Oct 31 21:15:18.510460 ip-10-0-136-157 systemd[1]: Starting Kubernetes Kubelet...
Oct 31 21:15:18.958963 ip-10-0-136-157 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 21:15:18.958963 ip-10-0-136-157 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 31 21:15:18.958963 ip-10-0-136-157 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 21:15:18.958963 ip-10-0-136-157 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 21:15:18.966553 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966466    2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 21:15:18.966553 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966548    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 21:15:18.966553 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966553    2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 21:15:18.966553 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966556    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 21:15:18.966553 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966559    2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 21:15:18.966553 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966562    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 21:15:18.966553 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966565    2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 21:15:18.966798 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966568    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 21:15:18.966798 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966571    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 21:15:18.966798 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966574    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 21:15:18.966798 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966576    2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 21:15:18.966798 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966579    2575 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 21:15:18.966798 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966581    2575 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 21:15:18.966798 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966584    2575 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 21:15:18.966798 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966586    2575 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 21:15:18.966798 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966589    2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 21:15:18.966798 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966592    2575 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 21:15:18.966798 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966594    2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 21:15:18.966798 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966597    2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 21:15:18.966798 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966599    2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 21:15:18.966798 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966602    2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 21:15:18.966798 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966604    2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 21:15:18.966798 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966607    2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 21:15:18.966798 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966610    2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 21:15:18.966798 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966612    2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 21:15:18.966798 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966615    2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 21:15:18.966798 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966617    2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 21:15:18.967277 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966626    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 21:15:18.967277 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966629    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 21:15:18.967277 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966631    2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 21:15:18.967277 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966634    2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 21:15:18.967277 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966636    2575 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 21:15:18.967277 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966639    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 21:15:18.967277 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966641    2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 21:15:18.967277 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966644    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 21:15:18.967277 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966647    2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 21:15:18.967277 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966650    2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 21:15:18.967277 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966652    2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 21:15:18.967277 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966655    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 21:15:18.967277 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966657    2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 21:15:18.967277 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966660    2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 21:15:18.967277 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966663    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 21:15:18.967277 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966666    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 21:15:18.967277 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966668    2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 21:15:18.967277 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966671    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 21:15:18.967277 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966674    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 21:15:18.967726 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966677    2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 21:15:18.967726 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966680    2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 21:15:18.967726 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966682    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 21:15:18.967726 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966685    2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 21:15:18.967726 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966689    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 21:15:18.967726 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966692    2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 21:15:18.967726 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966694    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 21:15:18.967726 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966697    2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 21:15:18.967726 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966699    2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 21:15:18.967726 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966702    2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 21:15:18.967726 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966704    2575 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 21:15:18.967726 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966707    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 21:15:18.967726 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966710    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 21:15:18.967726 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966712    2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 21:15:18.967726 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966715    2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 21:15:18.967726 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966717    2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 21:15:18.967726 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966720    2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 21:15:18.967726 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966722    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 21:15:18.967726 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966725    2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 21:15:18.968168 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966727    2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 21:15:18.968168 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966730    2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 21:15:18.968168 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966733    2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 21:15:18.968168 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966735    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 21:15:18.968168 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966737    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 21:15:18.968168 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966740    2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 21:15:18.968168 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966743    2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 21:15:18.968168 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966749    2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 21:15:18.968168 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966753    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 21:15:18.968168 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966755    2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 21:15:18.968168 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966758    2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 21:15:18.968168 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966761    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 21:15:18.968168 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966763    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 21:15:18.968168 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966766    2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 21:15:18.968168 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966769    2575 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 21:15:18.968168 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966771    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 21:15:18.968168 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966774    2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 21:15:18.968168 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966777    2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 21:15:18.968168 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966780    2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 21:15:18.968168 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966782    2575 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 21:15:18.968668 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966785    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 21:15:18.968668 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966787    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 21:15:18.968668 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966791    2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 21:15:18.968668 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966796    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 21:15:18.968668 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966800    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 21:15:18.968668 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966803    2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 21:15:18.968668 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966806    2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 21:15:18.968668 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966809    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 21:15:18.968668 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966812    2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 21:15:18.968668 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966815    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 21:15:18.968668 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966817    2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 21:15:18.968668 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966820    2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 21:15:18.968668 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966822    2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 21:15:18.968668 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966825    2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 21:15:18.968668 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966827    2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 21:15:18.968668 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966830    2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 21:15:18.968668 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966832    2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 21:15:18.968668 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966835    2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 21:15:18.968668 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966838    2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 21:15:18.969107 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966842    2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 21:15:18.969107 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.966844    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 21:15:18.969107 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967629    2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 21:15:18.969107 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967635    2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 21:15:18.969107 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967638    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 21:15:18.969107 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967641    2575 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 21:15:18.969107 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967644    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 21:15:18.969107 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967647    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 21:15:18.969107 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967649    2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 21:15:18.969107 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967652    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 21:15:18.969107 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967655    2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 21:15:18.969107 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967657    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 21:15:18.969107 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967660    2575 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 21:15:18.969107 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967663    2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 21:15:18.969107 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967666    2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 21:15:18.969107 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967669    2575 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 21:15:18.969107 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967671    2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 21:15:18.969107 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967673    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 21:15:18.969107 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967676    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 21:15:18.969107 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967678    2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 21:15:18.969628 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967681    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 21:15:18.969628 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967683    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 21:15:18.969628 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967686    2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 21:15:18.969628 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967689    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 21:15:18.969628 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967691    2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 21:15:18.969628 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967694    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 21:15:18.969628 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967696    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 21:15:18.969628 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967698    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 21:15:18.969628 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967701    2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 21:15:18.969628 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967703    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 21:15:18.969628 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967706    2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 21:15:18.969628 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967709    2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 21:15:18.969628 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967712    2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 21:15:18.969628 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967715    2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 21:15:18.969628 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967718    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 21:15:18.969628 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967721    2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 21:15:18.969628 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967723    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 21:15:18.969628 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967726    2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 21:15:18.969628 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967728    2575 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 21:15:18.970123 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967731    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 21:15:18.970123 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967734    2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 21:15:18.970123 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967736    2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 21:15:18.970123 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967739    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 21:15:18.970123 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967741    2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 21:15:18.970123 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967744    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 21:15:18.970123 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967747    2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 21:15:18.970123 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967750    2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 21:15:18.970123 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967752    2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 21:15:18.970123 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967755    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 21:15:18.970123 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967757    2575 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 21:15:18.970123 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967760    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 21:15:18.970123 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967762    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 21:15:18.970123 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967765    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 21:15:18.970123 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967767    2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 21:15:18.970123 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967770    2575 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 21:15:18.970123 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967772    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 21:15:18.970123 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967775    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 21:15:18.970123 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967777    2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 21:15:18.970593 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967780    2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 21:15:18.970593 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967782    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 21:15:18.970593 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967785    2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 21:15:18.970593 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967787    2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 21:15:18.970593 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967790    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 21:15:18.970593 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967793    2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 21:15:18.970593 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967796    2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 21:15:18.970593 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967799    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 21:15:18.970593 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967802    2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 21:15:18.970593 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967805    2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 21:15:18.970593 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967807    2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 21:15:18.970593 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967810    2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 21:15:18.970593 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967812    2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 21:15:18.970593 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967815    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 21:15:18.970593 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967817    2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 21:15:18.970593 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967820    2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 21:15:18.970593 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967822    2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 21:15:18.970593 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967825    2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 21:15:18.970593 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967827    2575 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 21:15:18.970593 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967829    2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 21:15:18.971054 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967832    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 21:15:18.971054 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967835    2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 21:15:18.971054 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967837    2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 21:15:18.971054 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967840    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 21:15:18.971054 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967842    2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 21:15:18.971054 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967845    2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 21:15:18.971054 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967847    2575 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 21:15:18.971054 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967850    2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 21:15:18.971054 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967852    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 21:15:18.971054 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967855    2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 21:15:18.971054 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967857    2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 21:15:18.971054 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967862    2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 21:15:18.971054 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967865    2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 21:15:18.971054 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967868    2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 21:15:18.971054 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967871    2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 21:15:18.971054 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967874    2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 21:15:18.971054 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967876    2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 21:15:18.971054 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967879    2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 21:15:18.971054 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967882    2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 21:15:18.971543 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967885    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 21:15:18.971543 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967888    2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 21:15:18.971543 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967892    2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 21:15:18.971543 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967895    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 21:15:18.971543 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967898    2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 21:15:18.971543 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967901    2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 21:15:18.971543 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967903    2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 21:15:18.971543 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967906    2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 21:15:18.971543 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967908    2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 21:15:18.971543 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967911    2575 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 21:15:18.971543 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.967914    2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 21:15:18.971543 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968426    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 21:15:18.971543 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968431    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 21:15:18.971543 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968434    2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 21:15:18.971543 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968437    2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 21:15:18.971543 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968440    2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 21:15:18.971543 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968443    2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 21:15:18.971543 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968445    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 21:15:18.971543 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968449    2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 21:15:18.972006 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968453    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 21:15:18.972006 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968457    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 21:15:18.972006 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968460    2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 21:15:18.972006 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968463    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 21:15:18.972006 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968466    2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 21:15:18.972006 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968468    2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 21:15:18.972006 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968471    2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 21:15:18.972006 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968474    2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 21:15:18.972006 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968477    2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 21:15:18.972006 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968480    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 21:15:18.972006 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968482    2575 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 21:15:18.972006 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968485    2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 21:15:18.972006 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968487    2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 21:15:18.972006 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968490    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 21:15:18.972006 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968492    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 21:15:18.972006 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968495    2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 21:15:18.972006 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968498    2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 21:15:18.972006 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968500    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 21:15:18.972006 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968502    2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 21:15:18.972006 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968505    2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 21:15:18.972484 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968508    2575 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 21:15:18.972484 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968511    2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 21:15:18.972484 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968513    2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 21:15:18.972484 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968516    2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 21:15:18.972484 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968518    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 21:15:18.972484 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968521    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 21:15:18.972484 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968523    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 21:15:18.972484 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968526    2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 21:15:18.972484 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968529    2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 21:15:18.972484 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968531    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 21:15:18.972484 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968534    2575 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 21:15:18.972484 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968537    2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 21:15:18.972484 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968539    2575 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 21:15:18.972484 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968542    2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 21:15:18.972484 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968545    2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 21:15:18.972484 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968548    2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 21:15:18.972484 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968551    2575 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 21:15:18.972484 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968553    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 21:15:18.972484 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968556    2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 21:15:18.972484 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968559    2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 21:15:18.972484 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968561    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 21:15:18.972986 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968564    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 21:15:18.972986 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968566    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 21:15:18.972986 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968569    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 21:15:18.972986 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968572    2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 21:15:18.972986 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968574    2575 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 21:15:18.972986 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968577    2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 21:15:18.972986 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968579    2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 21:15:18.972986 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968581    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 21:15:18.972986 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968584    2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 21:15:18.972986 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968586    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 21:15:18.972986 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968589    2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 21:15:18.972986 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968591    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 21:15:18.972986 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968594    2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 21:15:18.972986 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968596    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 21:15:18.972986 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968599    2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 21:15:18.972986 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968601    2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 21:15:18.972986 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968604    2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 21:15:18.972986 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968606    2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 21:15:18.972986 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968609    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 21:15:18.973448 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968612    2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 21:15:18.973448 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968615    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 21:15:18.973448 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968618    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 21:15:18.973448 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968620    2575 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 21:15:18.973448 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968623    2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 21:15:18.973448 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968625    2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 21:15:18.973448 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968628    2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 21:15:18.973448 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968633    2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 21:15:18.973448 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968637    2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 21:15:18.973448 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968640    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 21:15:18.973448 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968643    2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 21:15:18.973448 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968646    2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 21:15:18.973448 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968649    2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 21:15:18.973448 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968652    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 21:15:18.973448 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968654    2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 21:15:18.973448 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968657    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 21:15:18.973448 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968660    2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 21:15:18.973448 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968664    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 21:15:18.973448 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968667    2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 21:15:18.973892 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968669    2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 21:15:18.973892 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968672    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 21:15:18.973892 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968675    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 21:15:18.973892 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968677    2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 21:15:18.973892 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968680    2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 21:15:18.973892 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968683    2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 21:15:18.973892 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968685    2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 21:15:18.973892 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968688    2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 21:15:18.973892 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968691    2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 21:15:18.973892 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968693    2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 21:15:18.973892 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968696    2575 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 21:15:18.973892 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968698    2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 21:15:18.973892 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968701    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 21:15:18.973892 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968705    2575 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 21:15:18.973892 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968710    2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 21:15:18.973892 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968712    2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 21:15:18.973892 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968715    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 21:15:18.973892 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968718    2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 21:15:18.973892 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.968720    2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 21:15:18.974378 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969583    2575 flags.go:64] FLAG: --address="0.0.0.0"
Oct 31 21:15:18.974378 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969592    2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 31 21:15:18.974378 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969598    2575 flags.go:64] FLAG: --anonymous-auth="true"
Oct 31 21:15:18.974378 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969603    2575 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 31 21:15:18.974378 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969607    2575 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 31 21:15:18.974378 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969611    2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 31 21:15:18.974378 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969615    2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 31 21:15:18.974378 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969619    2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 31 21:15:18.974378 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969622    2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 31 21:15:18.974378 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969625    2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 31 21:15:18.974378 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969630    2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 31 21:15:18.974378 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969633    2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 31 21:15:18.974378 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969636    2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 31 21:15:18.974378 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969639    2575 flags.go:64] FLAG: --cgroup-root=""
Oct 31 21:15:18.974378 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969642    2575 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 31 21:15:18.974378 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969645    2575 flags.go:64] FLAG: --client-ca-file=""
Oct 31 21:15:18.974378 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969648    2575 flags.go:64] FLAG: --cloud-provider="external"
Oct 31 21:15:18.974378 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969651    2575 flags.go:64] FLAG: --cluster-dns="[]"
Oct 31 21:15:18.974378 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969657    2575 flags.go:64] FLAG: --cluster-domain=""
Oct 31 21:15:18.974378 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969660    2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 31 21:15:18.974378 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969663    2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 31 21:15:18.974378 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969666    2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 31 21:15:18.974378 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969670    2575 flags.go:64] FLAG: --container-log-max-files="5"
Oct 31 21:15:18.974378 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969674    2575 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 31 21:15:18.974930 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969677    2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 31 21:15:18.974930 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969680    2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 31 21:15:18.974930 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969684    2575 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 31 21:15:18.974930 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969687    2575 flags.go:64] FLAG: --contention-profiling="false"
Oct 31 21:15:18.974930 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969691    2575 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 31 21:15:18.974930 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969693    2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 31 21:15:18.974930 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969697    2575 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 31 21:15:18.974930 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969709    2575 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 31 21:15:18.974930 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969713    2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 31 21:15:18.974930 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969716    2575 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 31 21:15:18.974930 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969719    2575 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 31 21:15:18.974930 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969723    2575 flags.go:64] FLAG: --enable-load-reader="false"
Oct 31 21:15:18.974930 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969726    2575 flags.go:64] FLAG: --enable-server="true"
Oct 31 21:15:18.974930 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969728    2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 31 21:15:18.974930 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969733    2575 flags.go:64] FLAG: --event-burst="100"
Oct 31 21:15:18.974930 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969736    2575 flags.go:64] FLAG: --event-qps="50"
Oct 31 21:15:18.974930 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969739    2575 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 31 21:15:18.974930 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969742    2575 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 31 21:15:18.974930 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969745    2575 flags.go:64] FLAG: --eviction-hard=""
Oct 31 21:15:18.974930 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969749    2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 31 21:15:18.974930 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969752    2575 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 31 21:15:18.974930 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969755    2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 31 21:15:18.974930 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969758    2575 flags.go:64] FLAG: --eviction-soft=""
Oct 31 21:15:18.974930 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969761    2575 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 31 21:15:18.974930 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969765    2575 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 31 21:15:18.975686 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969769    2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 31 21:15:18.975686 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969772    2575 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 31 21:15:18.975686 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969774    2575 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 31 21:15:18.975686 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969777    2575 flags.go:64] FLAG: --fail-swap-on="true"
Oct 31 21:15:18.975686 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969780    2575 flags.go:64] FLAG: --feature-gates=""
Oct 31 21:15:18.975686 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969784    2575 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 31 21:15:18.975686 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969787    2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 31 21:15:18.975686 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969790    2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 31 21:15:18.975686 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969793    2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 31 21:15:18.975686 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969796    2575 flags.go:64] FLAG: --healthz-port="10248"
Oct 31 21:15:18.975686 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969800    2575 flags.go:64] FLAG: --help="false"
Oct 31 21:15:18.975686 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969802    2575 flags.go:64] FLAG: --hostname-override="ip-10-0-136-157.ec2.internal"
Oct 31 21:15:18.975686 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969806    2575 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 31 21:15:18.975686 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969810    2575 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 31 21:15:18.975686 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969814    2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 31 21:15:18.975686 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969817    2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 31 21:15:18.975686 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969820    2575 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 31 21:15:18.975686 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969823    2575 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 31 21:15:18.975686 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969826    2575 flags.go:64] FLAG: --image-service-endpoint=""
Oct 31 21:15:18.975686 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969829    2575 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 31 21:15:18.975686 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969832    2575 flags.go:64] FLAG: --kube-api-burst="100"
Oct 31 21:15:18.975686 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969835    2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 31 21:15:18.975686 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969838    2575 flags.go:64] FLAG: --kube-api-qps="50"
Oct 31 21:15:18.975686 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969840    2575 flags.go:64] FLAG: --kube-reserved=""
Oct 31 21:15:18.976283 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969843    2575 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 31 21:15:18.976283 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969846    2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 31 21:15:18.976283 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969849    2575 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 31 21:15:18.976283 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969852    2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 31 21:15:18.976283 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969855    2575 flags.go:64] FLAG: --lock-file=""
Oct 31 21:15:18.976283 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969858    2575 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 31 21:15:18.976283 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969860    2575 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 31 21:15:18.976283 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969864    2575 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 31 21:15:18.976283 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969869    2575 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 31 21:15:18.976283 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969872    2575 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 31 21:15:18.976283 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969875    2575 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 31 21:15:18.976283 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969878    2575 flags.go:64] FLAG: --logging-format="text"
Oct 31 21:15:18.976283 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969881    2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 31 21:15:18.976283 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969884    2575 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 31 21:15:18.976283 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969887    2575 flags.go:64] FLAG: --manifest-url=""
Oct 31 21:15:18.976283 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969889    2575 flags.go:64] FLAG: --manifest-url-header=""
Oct 31 21:15:18.976283 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969894    2575 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 31 21:15:18.976283 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969897    2575 flags.go:64] FLAG: --max-open-files="1000000"
Oct 31 21:15:18.976283 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969901    2575 flags.go:64] FLAG: --max-pods="110"
Oct 31 21:15:18.976283 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969904    2575 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 31 21:15:18.976283 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969907    2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 31 21:15:18.976283 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969911    2575 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 31 21:15:18.976283 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969915    2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 31 21:15:18.976283 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969918    2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 31 21:15:18.976283 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969921    2575 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 31 21:15:18.977190 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969924    2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 31 21:15:18.977190 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969931    2575 flags.go:64] FLAG: --node-status-max-images="50"
Oct 31 21:15:18.977190 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969934    2575 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 31 21:15:18.977190 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969937    2575 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 31 21:15:18.977190 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969940    2575 flags.go:64] FLAG: --pod-cidr=""
Oct 31 21:15:18.977190 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969943    2575 flags.go:64] FLAG: --pod-manifest-path=""
Oct 31 21:15:18.977190 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969946    2575 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 31 21:15:18.977190 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969948    2575 flags.go:64] FLAG: --pods-per-core="0"
Oct 31 21:15:18.977190 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969951    2575 flags.go:64] FLAG: --port="10250"
Oct 31 21:15:18.977190 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969954    2575 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 31 21:15:18.977190 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969957    2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-067941a7bcb0aa0b3"
Oct 31 21:15:18.977190 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969960    2575 flags.go:64] FLAG: --qos-reserved=""
Oct 31 21:15:18.977190 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969963    2575 flags.go:64] FLAG: --read-only-port="10255"
Oct 31 21:15:18.977190 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969966    2575 flags.go:64] FLAG: --register-node="true"
Oct 31 21:15:18.977190 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969969    2575 flags.go:64] FLAG: --register-with-taints=""
Oct 31 21:15:18.977190 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969973    2575 flags.go:64] FLAG: --registry-burst="10"
Oct 31 21:15:18.977190 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969976    2575 flags.go:64] FLAG: --registry-qps="5"
Oct 31 21:15:18.977190 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969979    2575 flags.go:64] FLAG: --reserved-cpus=""
Oct 31 21:15:18.977190 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969981    2575 flags.go:64] FLAG: --reserved-memory=""
Oct 31 21:15:18.977190 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969985    2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 31 21:15:18.977190 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969988    2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 31 21:15:18.977190 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969991    2575 flags.go:64] FLAG: --rotate-certificates="false"
Oct 31 21:15:18.977190 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969994    2575 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 31 21:15:18.977190 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.969997    2575 flags.go:64] FLAG: --runonce="false"
Oct 31 21:15:18.977190 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.970000    2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 31 21:15:18.977190 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.970003    2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 31 21:15:18.978319 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.970006    2575 flags.go:64] FLAG: --seccomp-default="false"
Oct 31 21:15:18.978319 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.970009    2575 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 31 21:15:18.978319 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.970011    2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 31 21:15:18.978319 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.970016    2575 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 31 21:15:18.978319 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.970020    2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 31 21:15:18.978319 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.970023    2575 flags.go:64] FLAG: --storage-driver-password="root"
Oct 31 21:15:18.978319 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.970026    2575 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 31 21:15:18.978319 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.970028    2575 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 31 21:15:18.978319 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.970031    2575 flags.go:64] FLAG: --storage-driver-user="root"
Oct 31 21:15:18.978319 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.970034    2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 31 21:15:18.978319 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.970037    2575 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 31 21:15:18.978319 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.970040    2575 flags.go:64] FLAG: --system-cgroups=""
Oct 31 21:15:18.978319 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.970042    2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 31 21:15:18.978319 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.970047    2575 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 31 21:15:18.978319 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.970050    2575 flags.go:64] FLAG: --tls-cert-file=""
Oct 31 21:15:18.978319 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.970053    2575 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 31 21:15:18.978319 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.970058    2575 flags.go:64] FLAG: --tls-min-version=""
Oct 31 21:15:18.978319 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.970061    2575 flags.go:64] FLAG: --tls-private-key-file=""
Oct 31 21:15:18.978319 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.970063    2575 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 31 21:15:18.978319 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.970066    2575 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 31 21:15:18.978319 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.970069    2575 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 31 21:15:18.978319 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.970072    2575 flags.go:64] FLAG: --v="2"
Oct 31 21:15:18.978319 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.970076    2575 flags.go:64] FLAG: --version="false"
Oct 31 21:15:18.978319 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.970081    2575 flags.go:64] FLAG: --vmodule=""
Oct 31 21:15:18.978319 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.970084    2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 31 21:15:18.979372 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.970087    2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 31 21:15:18.979372 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970232    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 21:15:18.979372 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970236    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 21:15:18.979372 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970239    2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 21:15:18.979372 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970242    2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 21:15:18.979372 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970246    2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 21:15:18.979372 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970249    2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 21:15:18.979372 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970252    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 21:15:18.979372 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970255    2575 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 21:15:18.979372 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970258    2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 21:15:18.979372 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970262    2575 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 21:15:18.979372 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970266    2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 21:15:18.979372 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970268    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 21:15:18.979372 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970271    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 21:15:18.979372 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970274    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 21:15:18.979372 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970277    2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 21:15:18.979372 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970279    2575 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 21:15:18.979372 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970282    2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 21:15:18.979372 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970284    2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 21:15:18.979372 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970287    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 21:15:18.980261 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970289    2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 21:15:18.980261 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970292    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 21:15:18.980261 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970295    2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 21:15:18.980261 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970297    2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 21:15:18.980261 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970300    2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 21:15:18.980261 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970302    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 21:15:18.980261 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970305    2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 21:15:18.980261 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970307    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 21:15:18.980261 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970310    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 21:15:18.980261 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970312    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 21:15:18.980261 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970315    2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 21:15:18.980261 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970317    2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 21:15:18.980261 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970320    2575 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 21:15:18.980261 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970322    2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 21:15:18.980261 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970325    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 21:15:18.980261 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970327    2575 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 21:15:18.980261 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970330    2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 21:15:18.980261 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970335    2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 21:15:18.980261 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970338    2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 21:15:18.981080 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970341    2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 21:15:18.981080 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970343    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 21:15:18.981080 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970346    2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 21:15:18.981080 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970350    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 21:15:18.981080 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970352    2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 21:15:18.981080 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970364    2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 21:15:18.981080 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970367    2575 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 21:15:18.981080 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970369    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 21:15:18.981080 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970372    2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 21:15:18.981080 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970374    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 21:15:18.981080 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970377    2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 21:15:18.981080 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970381    2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 21:15:18.981080 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970384    2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 21:15:18.981080 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970387    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 21:15:18.981080 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970389    2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 21:15:18.981080 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970392    2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 21:15:18.981080 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970395    2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 21:15:18.981080 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970397    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 21:15:18.981080 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970400    2575 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 21:15:18.981564 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970402    2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 21:15:18.981564 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970405    2575 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 21:15:18.981564 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970407    2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 21:15:18.981564 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970410    2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 21:15:18.981564 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970413    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 21:15:18.981564 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970415    2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 21:15:18.981564 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970418    2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 21:15:18.981564 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970420    2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 21:15:18.981564 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970423    2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 21:15:18.981564 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970425    2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 21:15:18.981564 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970428    2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 21:15:18.981564 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970432    2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 21:15:18.981564 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970435    2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 21:15:18.981564 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970437    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 21:15:18.981564 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970440    2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 21:15:18.981564 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970442    2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 21:15:18.981564 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970446    2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 21:15:18.981564 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970449    2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 21:15:18.981564 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970451    2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 21:15:18.981564 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970454    2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 21:15:18.982280 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970457    2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 21:15:18.982280 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970460    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 21:15:18.982280 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970463    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 21:15:18.982280 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970465    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 21:15:18.982280 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970468    2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 21:15:18.982280 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970470    2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 21:15:18.982280 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970473    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 21:15:18.982280 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970475    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 21:15:18.982280 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970478    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 21:15:18.982280 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970480    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 21:15:18.982280 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970483    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 21:15:18.982280 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970485    2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 21:15:18.982280 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970488    2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 21:15:18.982280 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970490    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 21:15:18.982280 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970493    2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 21:15:18.982280 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970495    2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 21:15:18.982280 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970498    2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 21:15:18.982280 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970500    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 21:15:18.982280 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970502    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 21:15:18.982280 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970505    2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 21:15:18.983143 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970508    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 21:15:18.983143 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970510    2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 21:15:18.983143 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970514    2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 21:15:18.983143 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970520    2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 21:15:18.983143 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970523    2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 21:15:18.983143 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970526    2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 21:15:18.983143 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970528    2575 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 21:15:18.983143 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970531    2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 21:15:18.983143 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970535    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 21:15:18.983143 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.970563    2575 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 21:15:18.983143 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970603    2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 21:15:18.983143 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970608    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 21:15:18.983143 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970612    2575 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 21:15:18.983143 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970616    2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 21:15:18.983143 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970620    2575 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 21:15:18.983143 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970622    2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 21:15:18.983677 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970625    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 21:15:18.983677 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970627    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 21:15:18.983677 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970630    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 21:15:18.983677 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970633    2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 21:15:18.983677 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970636    2575 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 21:15:18.983677 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970638    2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 21:15:18.983677 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970641    2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 21:15:18.983677 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970643    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 21:15:18.983677 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970646    2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 21:15:18.983677 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970648    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 21:15:18.983677 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970651    2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 21:15:18.983677 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970653    2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 21:15:18.983677 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970656    2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 21:15:18.983677 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970658    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 21:15:18.983677 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970661    2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 21:15:18.983677 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970664    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 21:15:18.983677 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970666    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 21:15:18.983677 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970669    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 21:15:18.983677 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970671    2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 21:15:18.984275 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970674    2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 21:15:18.984275 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970678    2575 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 21:15:18.984275 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970681    2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 21:15:18.984275 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970684    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 21:15:18.984275 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970686    2575 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 21:15:18.984275 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970690    2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 21:15:18.984275 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970692    2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 21:15:18.984275 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970695    2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 21:15:18.984275 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970698    2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 21:15:18.984275 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970700    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 21:15:18.984275 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970703    2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 21:15:18.984275 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970706    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 21:15:18.984275 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970708    2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 21:15:18.984275 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970711    2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 21:15:18.984275 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970713    2575 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 21:15:18.984275 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970716    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 21:15:18.984275 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970719    2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 21:15:18.984275 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970721    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 21:15:18.984275 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970724    2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 21:15:18.984721 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970728    2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 21:15:18.984721 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970731    2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 21:15:18.984721 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970734    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 21:15:18.984721 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970737    2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 21:15:18.984721 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970739    2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 21:15:18.984721 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970742    2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 21:15:18.984721 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970745    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 21:15:18.984721 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970747    2575 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 21:15:18.984721 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970750    2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 21:15:18.984721 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970752    2575 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 21:15:18.984721 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970755    2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 21:15:18.984721 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970757    2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 21:15:18.984721 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970760    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 21:15:18.984721 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970762    2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 21:15:18.984721 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970766    2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 21:15:18.984721 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970769    2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 21:15:18.984721 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970771    2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 21:15:18.984721 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970774    2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 21:15:18.984721 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970776    2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 21:15:18.985239 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970779    2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 21:15:18.985239 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970781    2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 21:15:18.985239 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970784    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 21:15:18.985239 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970787    2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 21:15:18.985239 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970789    2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 21:15:18.985239 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970793    2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 21:15:18.985239 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970796    2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 21:15:18.985239 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970798    2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 21:15:18.985239 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970801    2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 21:15:18.985239 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970804    2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 21:15:18.985239 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970806    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 21:15:18.985239 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970809    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 21:15:18.985239 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970811    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 21:15:18.985239 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970814    2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 21:15:18.985239 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970817    2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 21:15:18.985239 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970819    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 21:15:18.985239 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970822    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 21:15:18.985239 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970824    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 21:15:18.985239 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970827    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 21:15:18.985239 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970829    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 21:15:18.985697 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970832    2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 21:15:18.985697 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970834    2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 21:15:18.985697 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970837    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 21:15:18.985697 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970839    2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 21:15:18.985697 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970842    2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 21:15:18.985697 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970845    2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 21:15:18.985697 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970847    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 21:15:18.985697 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970851    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 21:15:18.985697 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970853    2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 21:15:18.985697 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970856    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 21:15:18.985697 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970858    2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 21:15:18.985697 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970863    2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 21:15:18.985697 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970866    2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 21:15:18.985697 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970869    2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 21:15:18.985697 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970872    2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 21:15:18.985697 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970875    2575 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 21:15:18.985697 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970877    2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 21:15:18.985697 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970880    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 21:15:18.985697 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970883    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 21:15:18.986342 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970886    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 21:15:18.986342 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970888    2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 21:15:18.986342 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970891    2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 21:15:18.986342 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.970893    2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 21:15:18.986342 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.978968    2575 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 31 21:15:18.986342 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.978990    2575 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 31 21:15:18.986342 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.979014    2575 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 31 21:15:18.986342 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.979021    2575 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 31 21:15:18.986342 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979103    2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 21:15:18.986342 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979111    2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 21:15:18.986342 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979116    2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 21:15:18.986342 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979121    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 21:15:18.986342 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979126    2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 21:15:18.986342 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979131    2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 21:15:18.986342 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979135    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 21:15:18.986342 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979139    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 21:15:18.986342 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979143    2575 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 21:15:18.986342 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979148    2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 21:15:18.986342 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979152    2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 21:15:18.986804 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979156    2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 21:15:18.986804 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979161    2575 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 21:15:18.986804 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979165    2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 21:15:18.986804 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979169    2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 21:15:18.986804 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979189    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 21:15:18.986804 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979195    2575 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 21:15:18.986804 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979199    2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 21:15:18.986804 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979203    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 21:15:18.986804 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979207    2575 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 21:15:18.986804 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979212    2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 21:15:18.986804 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979217    2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 21:15:18.986804 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979221    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 21:15:18.986804 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979226    2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 21:15:18.986804 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979230    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 21:15:18.986804 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979234    2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 21:15:18.986804 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979238    2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 21:15:18.986804 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979242    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 21:15:18.986804 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979246    2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 21:15:18.986804 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979253    2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 21:15:18.986804 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979257    2575 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 21:15:18.987528 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979262    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 21:15:18.987528 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979266    2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 21:15:18.987528 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979271    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 21:15:18.987528 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979275    2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 21:15:18.987528 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979279    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 21:15:18.987528 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979283    2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 21:15:18.987528 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979287    2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 21:15:18.987528 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979291    2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 21:15:18.987528 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979296    2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 21:15:18.987528 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979300    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 21:15:18.987528 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979304    2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 21:15:18.987528 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979311    2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 21:15:18.987528 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979318    2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 21:15:18.987528 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979323    2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 21:15:18.987528 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979327    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 21:15:18.987528 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979331    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 21:15:18.987528 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979337    2575 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 21:15:18.987528 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979341    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 21:15:18.987528 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979345    2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 21:15:18.988317 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979350    2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 21:15:18.988317 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979354    2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 21:15:18.988317 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979358    2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 21:15:18.988317 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979363    2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 21:15:18.988317 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979367    2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 21:15:18.988317 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979371    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 21:15:18.988317 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979375    2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 21:15:18.988317 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979379    2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 21:15:18.988317 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979382    2575 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 21:15:18.988317 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979386    2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 21:15:18.988317 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979390    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 21:15:18.988317 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979395    2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 21:15:18.988317 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979400    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 21:15:18.988317 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979404    2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 21:15:18.988317 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979409    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 21:15:18.988317 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979413    2575 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 21:15:18.988317 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979417    2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 21:15:18.988317 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979421    2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 21:15:18.988317 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979426    2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 21:15:18.988317 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979430    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 21:15:18.989071 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979434    2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 21:15:18.989071 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979439    2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 21:15:18.989071 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979443    2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 21:15:18.989071 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979447    2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 21:15:18.989071 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979451    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 21:15:18.989071 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979455    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 21:15:18.989071 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979460    2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 21:15:18.989071 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979464    2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 21:15:18.989071 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979468    2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 21:15:18.989071 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979473    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 21:15:18.989071 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979477    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 21:15:18.989071 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979481    2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 21:15:18.989071 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979485    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 21:15:18.989071 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979489    2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 21:15:18.989071 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979495    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 21:15:18.989071 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979499    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 21:15:18.989071 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979503    2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 21:15:18.989071 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979508    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 21:15:18.989071 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979512    2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 21:15:18.989071 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979516    2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 21:15:18.989599 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979520    2575 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 21:15:18.989599 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979524    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 21:15:18.989599 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979528    2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 21:15:18.989599 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979532    2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 21:15:18.989599 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979537    2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 21:15:18.989599 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979544    2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 21:15:18.989599 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979550    2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 21:15:18.989599 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979554    2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 21:15:18.989599 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979558    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 21:15:18.989599 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979563    2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 21:15:18.989599 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979567    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 21:15:18.989599 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979571    2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 21:15:18.989599 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979576    2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 21:15:18.989599 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979580    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 21:15:18.989599 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979584    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 21:15:18.989599 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979588    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 21:15:18.990023 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.979629    2575 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 21:15:18.990023 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979850    2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 21:15:18.990023 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979857    2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 21:15:18.990023 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979862    2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 21:15:18.990023 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979867    2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 21:15:18.990023 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979871    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 21:15:18.990023 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979875    2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 21:15:18.990023 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979879    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 21:15:18.990023 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979884    2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 21:15:18.990023 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979888    2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 21:15:18.990023 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979892    2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 21:15:18.990023 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979897    2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 21:15:18.990023 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979901    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 21:15:18.990023 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979905    2575 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 21:15:18.990023 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979910    2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 21:15:18.990023 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979914    2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 21:15:18.990452 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979918    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 21:15:18.990452 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979922    2575 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 21:15:18.990452 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979926    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 21:15:18.990452 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979931    2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 21:15:18.990452 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979935    2575 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 21:15:18.990452 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979939    2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 21:15:18.990452 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979944    2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 21:15:18.990452 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979948    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 21:15:18.990452 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979953    2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 21:15:18.990452 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979957    2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 21:15:18.990452 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979961    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 21:15:18.990452 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979965    2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 21:15:18.990452 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979969    2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 21:15:18.990452 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979973    2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 21:15:18.990452 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979977    2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 21:15:18.990452 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979981    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 21:15:18.990452 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979986    2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 21:15:18.990452 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979990    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 21:15:18.990452 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979994    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 21:15:18.990452 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.979998    2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 21:15:18.990914 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980002    2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 21:15:18.990914 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980006    2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 21:15:18.990914 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980010    2575 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 21:15:18.990914 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980015    2575 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 21:15:18.990914 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980019    2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 21:15:18.990914 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980023    2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 21:15:18.990914 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980027    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 21:15:18.990914 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980031    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 21:15:18.990914 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980036    2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 21:15:18.990914 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980040    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 21:15:18.990914 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980044    2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 21:15:18.990914 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980048    2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 21:15:18.990914 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980052    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 21:15:18.990914 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980056    2575 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 21:15:18.990914 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980060    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 21:15:18.990914 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980064    2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 21:15:18.990914 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980068    2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 21:15:18.990914 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980073    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 21:15:18.990914 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980077    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 21:15:18.991410 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980084    2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 21:15:18.991410 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980091    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 21:15:18.991410 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980096    2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 21:15:18.991410 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980100    2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 21:15:18.991410 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980105    2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 21:15:18.991410 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980111    2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 21:15:18.991410 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980117    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 21:15:18.991410 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980123    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 21:15:18.991410 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980127    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 21:15:18.991410 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980131    2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 21:15:18.991410 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980135    2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 21:15:18.991410 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980139    2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 21:15:18.991410 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980143    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 21:15:18.991410 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980148    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 21:15:18.991410 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980152    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 21:15:18.991410 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980156    2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 21:15:18.991410 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980160    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 21:15:18.991410 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980165    2575 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 21:15:18.991410 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980169    2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 21:15:18.991907 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980191    2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 21:15:18.991907 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980196    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 21:15:18.991907 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980200    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 21:15:18.991907 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980205    2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 21:15:18.991907 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980209    2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 21:15:18.991907 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980213    2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 21:15:18.991907 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980217    2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 21:15:18.991907 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980222    2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 21:15:18.991907 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980226    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 21:15:18.991907 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980230    2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 21:15:18.991907 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980234    2575 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 21:15:18.991907 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980239    2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 21:15:18.991907 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980243    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 21:15:18.991907 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980247    2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 21:15:18.991907 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980252    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 21:15:18.991907 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980256    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 21:15:18.991907 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980260    2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 21:15:18.991907 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980264    2575 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 21:15:18.991907 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980268    2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 21:15:18.991907 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980272    2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 21:15:18.992517 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980276    2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 21:15:18.992517 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980280    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 21:15:18.992517 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980285    2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 21:15:18.992517 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980289    2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 21:15:18.992517 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980293    2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 21:15:18.992517 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980298    2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 21:15:18.992517 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980302    2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 21:15:18.992517 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980306    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 21:15:18.992517 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980310    2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 21:15:18.992517 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980315    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 21:15:18.992517 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980319    2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 21:15:18.992517 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980327    2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 21:15:18.992517 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980331    2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 21:15:18.992517 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.980372    2575 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 21:15:18.992517 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980424    2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 21:15:18.992517 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980432    2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 21:15:18.992903 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980437    2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 21:15:18.992903 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980441    2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 21:15:18.992903 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980446    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 21:15:18.992903 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980451    2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 21:15:18.992903 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980455    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 21:15:18.992903 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980460    2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 21:15:18.992903 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980465    2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 21:15:18.992903 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980469    2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 21:15:18.992903 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980473    2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 21:15:18.992903 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980478    2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 21:15:18.992903 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980482    2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 21:15:18.992903 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980487    2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 21:15:18.992903 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980491    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 21:15:18.992903 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980495    2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 21:15:18.992903 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980499    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 21:15:18.992903 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980504    2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 21:15:18.992903 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980508    2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 21:15:18.992903 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980512    2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 21:15:18.992903 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980516    2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 21:15:18.992903 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980520    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 21:15:18.993402 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980525    2575 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 21:15:18.993402 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980529    2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 21:15:18.993402 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980533    2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 21:15:18.993402 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980537    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 21:15:18.993402 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980541    2575 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 21:15:18.993402 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980546    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 21:15:18.993402 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980550    2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 21:15:18.993402 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980556    2575 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 21:15:18.993402 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980560    2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 21:15:18.993402 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980564    2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 21:15:18.993402 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980569    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 21:15:18.993402 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980573    2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 21:15:18.993402 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980577    2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 21:15:18.993402 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980581    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 21:15:18.993402 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980586    2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 21:15:18.993402 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980590    2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 21:15:18.993402 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980594    2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 21:15:18.993402 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980599    2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 21:15:18.993402 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980603    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 21:15:18.993402 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980607    2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 21:15:18.993876 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980611    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 21:15:18.993876 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980615    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 21:15:18.993876 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980620    2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 21:15:18.993876 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980623    2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 21:15:18.993876 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980628    2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 21:15:18.993876 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980633    2575 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 21:15:18.993876 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980636    2575 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 21:15:18.993876 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980641    2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 21:15:18.993876 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980645    2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 21:15:18.993876 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980649    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 21:15:18.993876 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980654    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 21:15:18.993876 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980658    2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 21:15:18.993876 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980662    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 21:15:18.993876 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980667    2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 21:15:18.993876 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980671    2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 21:15:18.993876 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980675    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 21:15:18.993876 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980679    2575 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 21:15:18.993876 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980683    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 21:15:18.993876 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980688    2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 21:15:18.993876 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980694    2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 21:15:18.994430 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980698    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 21:15:18.994430 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980703    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 21:15:18.994430 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980709    2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 21:15:18.994430 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980716    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 21:15:18.994430 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980722    2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 21:15:18.994430 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980727    2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 21:15:18.994430 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980731    2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 21:15:18.994430 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980737    2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 21:15:18.994430 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980743    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 21:15:18.994430 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980749    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 21:15:18.994430 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980754    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 21:15:18.994430 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980758    2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 21:15:18.994430 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980763    2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 21:15:18.994430 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980768    2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 21:15:18.994430 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980772    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 21:15:18.994430 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980776    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 21:15:18.994430 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980781    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 21:15:18.994430 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980786    2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 21:15:18.994849 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980790    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 21:15:18.994849 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980795    2575 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 21:15:18.994849 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980799    2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 21:15:18.994849 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980803    2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 21:15:18.994849 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980807    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 21:15:18.994849 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980811    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 21:15:18.994849 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980816    2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 21:15:18.994849 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980820    2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 21:15:18.994849 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980824    2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 21:15:18.994849 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980828    2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 21:15:18.994849 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980832    2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 21:15:18.994849 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980837    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 21:15:18.994849 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980841    2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 21:15:18.994849 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980847    2575 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 21:15:18.994849 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980851    2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 21:15:18.994849 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980855    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 21:15:18.994849 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980860    2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 21:15:18.994849 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980864    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 21:15:18.994849 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980868    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 21:15:18.995311 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980872    2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 21:15:18.995311 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980876    2575 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 21:15:18.995311 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980880    2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 21:15:18.995311 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980884    2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 21:15:18.995311 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980888    2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 21:15:18.995311 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980893    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 21:15:18.995311 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:18.980897    2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 21:15:18.995311 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.980975    2575 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 31 21:15:18.995311 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.983998    2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 31 21:15:18.995311 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.985130    2575 server.go:1014] "Starting client certificate rotation"
Oct 31 21:15:18.995311 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.985229    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 31 21:15:18.995311 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.985274    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 31 21:15:18.995311 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.987611    2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 21:15:18.995311 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:18.991208    2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 21:15:19.002847 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.002833    2575 log.go:25] "Validated CRI v1 runtime API"
Oct 31 21:15:19.008490 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.008478    2575 log.go:25] "Validated CRI v1 image API"
Oct 31 21:15:19.009347 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.009321    2575 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 31 21:15:19.012073 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.012054    2575 fs.go:135] Filesystem UUIDs: map[257b3f3f-5801-4157-8685-c44a82731d46:/dev/nvme0n1p4 5fee886e-9fef-4555-a7ff-01e4fa7789c5:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Oct 31 21:15:19.012154 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.012071    2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 31 21:15:19.017187 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.017065    2575 manager.go:217] Machine: {Timestamp:2026-10-31 21:15:19.015625912 +0000 UTC m=+0.377389558 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:2499998 MemoryCapacity:33164333056 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec221116a41121a82aa1d46fe2bf9f96 SystemUUID:ec221116-a411-21a8-2aa1-d46fe2bf9f96 BootID:c4974a24-3e9e-421a-af93-99a9bf8c0cca Filesystems:[{Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582164480 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:e0:b1:f1:3f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:e0:b1:f1:3f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f6:2a:a0:38:ef:8d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164333056 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 31 21:15:19.017187 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.017185    2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 31 21:15:19.017292 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.017286    2575 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 31 21:15:19.020984 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.020960    2575 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 31 21:15:19.021111 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.020987    2575 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-157.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 31 21:15:19.021783 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.021773    2575 topology_manager.go:143] "Creating topology manager with none policy"
Oct 31 21:15:19.021822 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.021784    2575 container_manager_linux.go:311] "Creating device plugin manager"
Oct 31 21:15:19.021822 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.021809    2575 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 21:15:19.021879 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.021823    2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 21:15:19.023011 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.023000    2575 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 31 21:15:19.027267 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.027255    2575 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 31 21:15:19.027378 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.027369    2575 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 31 21:15:19.028234 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.028225    2575 kubelet.go:493] "Attempting to sync node with API server"
Oct 31 21:15:19.028272 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.028238    2575 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 31 21:15:19.028272 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.028248    2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 31 21:15:19.028272 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.028257    2575 kubelet.go:405] "Adding apiserver pod source"
Oct 31 21:15:19.028272 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.028264    2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 31 21:15:19.028575 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.028563    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 21:15:19.034255 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.034238    2575 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 31 21:15:19.040703 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.040688    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 21:15:19.041514 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.041501    2575 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 31 21:15:19.041560 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.041532    2575 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 31 21:15:19.041696 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.041685    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 31 21:15:19.041733 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.041709    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 31 21:15:19.041733 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.041718    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 31 21:15:19.041733 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.041724    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 31 21:15:19.041733 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.041730    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 31 21:15:19.041832 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.041736    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 31 21:15:19.042356 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.042347    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 31 21:15:19.042388 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.042357    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 31 21:15:19.042388 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.042364    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 31 21:15:19.042388 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.042371    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 31 21:15:19.042388 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.042379    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 31 21:15:19.042388 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.042387    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 31 21:15:19.043113 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.043103    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 31 21:15:19.043143 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.043116    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 31 21:15:19.047124 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.047109    2575 server.go:1263] "Started kubelet"
Oct 31 21:15:19.047230 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.047204    2575 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 31 21:15:19.047276 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.047232    2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 31 21:15:19.047316 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.047290    2575 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 31 21:15:19.048570 ip-10-0-136-157 systemd[1]: Started Kubernetes Kubelet.
Oct 31 21:15:19.049544 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.048659    2575 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 31 21:15:19.051101 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.050566    2575 server.go:317] "Adding debug handlers to kubelet server"
Oct 31 21:15:19.054149 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:19.054125    2575 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 31 21:15:19.055941 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.055925    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 31 21:15:19.057298 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.057282    2575 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 31 21:15:19.057499 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:19.057475    2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-157.ec2.internal\" not found"
Oct 31 21:15:19.058072 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.058057    2575 factory.go:55] Registering systemd factory
Oct 31 21:15:19.058072 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.058073    2575 factory.go:223] Registration of the systemd container factory successfully
Oct 31 21:15:19.058234 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.058074    2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 31 21:15:19.058234 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.058077    2575 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 31 21:15:19.058234 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.058098    2575 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 31 21:15:19.058234 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.058210    2575 reconstruct.go:97] "Volume reconstruction finished"
Oct 31 21:15:19.058234 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.058219    2575 reconciler.go:29] "Reconciler: start to sync state"
Oct 31 21:15:19.058443 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.058298    2575 factory.go:153] Registering CRI-O factory
Oct 31 21:15:19.058443 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.058311    2575 factory.go:223] Registration of the crio container factory successfully
Oct 31 21:15:19.058443 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.058353    2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 31 21:15:19.058443 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.058368    2575 factory.go:103] Registering Raw factory
Oct 31 21:15:19.058443 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.058378    2575 manager.go:1199] Started watching for new ooms in manager
Oct 31 21:15:19.059368 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.059354    2575 manager.go:319] Starting recovery of all containers
Oct 31 21:15:19.073733 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.073714    2575 manager.go:324] Recovery completed
Oct 31 21:15:19.078368 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.078354    2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 21:15:19.079292 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.079251    2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-157.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 31 21:15:19.079383 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:19.079279    2575 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-157.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 31 21:15:19.079952 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.079933    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasSufficientMemory"
Oct 31 21:15:19.080022 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.079962    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasNoDiskPressure"
Oct 31 21:15:19.080022 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.079972    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasSufficientPID"
Oct 31 21:15:19.080396 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:19.079256    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-157.ec2.internal.18e3b9c1e519b99f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-157.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-157.ec2.internal,},FirstTimestamp:2026-10-31 21:15:19.047084447 +0000 UTC m=+0.408848094,LastTimestamp:2026-10-31 21:15:19.047084447 +0000 UTC m=+0.408848094,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-157.ec2.internal,}"
Oct 31 21:15:19.080534 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.080518    2575 cpu_manager.go:226] "Starting" policy="none"
Oct 31 21:15:19.080596 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.080533    2575 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 31 21:15:19.080596 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.080540    2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 21:15:19.080596 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.080552    2575 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 31 21:15:19.081251 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.081236    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasSufficientMemory"
Oct 31 21:15:19.081312 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.081263    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasNoDiskPressure"
Oct 31 21:15:19.081312 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.081272    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasSufficientPID"
Oct 31 21:15:19.082850 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.082836    2575 policy_none.go:50] "Start"
Oct 31 21:15:19.082850 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.082852    2575 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 31 21:15:19.082938 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.082862    2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 31 21:15:19.084847 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.084831    2575 policy_none.go:44] "Start"
Oct 31 21:15:19.094861 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:19.094832    2575 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-157.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 31 21:15:19.094957 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:19.094917    2575 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 31 21:15:19.095006 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:19.094979    2575 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 31 21:15:19.095144 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:19.095086    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-157.ec2.internal.18e3b9c1e70f351a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-157.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-157.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-157.ec2.internal,},FirstTimestamp:2026-10-31 21:15:19.079949594 +0000 UTC m=+0.441713243,LastTimestamp:2026-10-31 21:15:19.079949594 +0000 UTC m=+0.441713243,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-157.ec2.internal,}"
Oct 31 21:15:19.120100 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.120068    2575 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 31 21:15:19.120694 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:19.120624    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-157.ec2.internal.18e3b9c1e70f78e7  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-157.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-136-157.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-136-157.ec2.internal,},FirstTimestamp:2026-10-31 21:15:19.079966951 +0000 UTC m=+0.441730597,LastTimestamp:2026-10-31 21:15:19.079966951 +0000 UTC m=+0.441730597,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-157.ec2.internal,}"
Oct 31 21:15:19.122921 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.122904    2575 manager.go:341] "Starting Device Plugin manager"
Oct 31 21:15:19.123535 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:19.123519    2575 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 31 21:15:19.123594 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.123544    2575 server.go:87] "Starting device plugin registration server"
Oct 31 21:15:19.123746 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.123734    2575 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 31 21:15:19.123790 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.123746    2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 31 21:15:19.123863 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.123843    2575 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 31 21:15:19.124267 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.123941    2575 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 31 21:15:19.124267 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.123955    2575 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 31 21:15:19.124524 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:19.124495    2575 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 31 21:15:19.124614 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:19.124542    2575 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-157.ec2.internal\" not found"
Oct 31 21:15:19.150632 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:19.150561    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-157.ec2.internal.18e3b9c1e70f9e07  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-157.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-136-157.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-136-157.ec2.internal,},FirstTimestamp:2026-10-31 21:15:19.079976455 +0000 UTC m=+0.441740100,LastTimestamp:2026-10-31 21:15:19.079976455 +0000 UTC m=+0.441740100,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-157.ec2.internal,}"
Oct 31 21:15:19.168696 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.151783    2575 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 31 21:15:19.168696 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.151827    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nk2wv"
Oct 31 21:15:19.171634 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.168755    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nk2wv"
Oct 31 21:15:19.191937 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.191910    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 31 21:15:19.193044 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.193019    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 31 21:15:19.193113 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.193050    2575 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 31 21:15:19.193113 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.193074    2575 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 31 21:15:19.193229 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:19.193114    2575 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 31 21:15:19.209189 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.209131    2575 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 21:15:19.224324 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.224311    2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 21:15:19.225119 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.225104    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasSufficientMemory"
Oct 31 21:15:19.225205 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.225130    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasNoDiskPressure"
Oct 31 21:15:19.225205 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.225155    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasSufficientPID"
Oct 31 21:15:19.225205 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.225191    2575 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-136-157.ec2.internal"
Oct 31 21:15:19.245918 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.245898    2575 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-136-157.ec2.internal"
Oct 31 21:15:19.245996 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:19.245921    2575 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-157.ec2.internal\": node \"ip-10-0-136-157.ec2.internal\" not found"
Oct 31 21:15:19.288954 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:19.288936    2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-157.ec2.internal\" not found"
Oct 31 21:15:19.293962 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.293946    2575 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-157.ec2.internal"]
Oct 31 21:15:19.294024 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.293998    2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 21:15:19.294770 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.294754    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasSufficientMemory"
Oct 31 21:15:19.294826 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.294792    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasNoDiskPressure"
Oct 31 21:15:19.294826 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.294808    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasSufficientPID"
Oct 31 21:15:19.297082 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.297070    2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 21:15:19.297252 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.297238    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal"
Oct 31 21:15:19.297302 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.297292    2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 21:15:19.297847 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.297831    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasSufficientMemory"
Oct 31 21:15:19.297896 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.297863    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasNoDiskPressure"
Oct 31 21:15:19.297896 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.297873    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasSufficientPID"
Oct 31 21:15:19.297957 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.297836    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasSufficientMemory"
Oct 31 21:15:19.297957 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.297939    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasNoDiskPressure"
Oct 31 21:15:19.297957 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.297951    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasSufficientPID"
Oct 31 21:15:19.300067 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.300053    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-157.ec2.internal"
Oct 31 21:15:19.300134 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.300082    2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 21:15:19.300763 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.300742    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasSufficientMemory"
Oct 31 21:15:19.300833 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.300770    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasNoDiskPressure"
Oct 31 21:15:19.300833 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.300791    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasSufficientPID"
Oct 31 21:15:19.332040 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:19.332019    2575 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-157.ec2.internal\" not found" node="ip-10-0-136-157.ec2.internal"
Oct 31 21:15:19.335025 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:19.335004    2575 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-157.ec2.internal\" not found" node="ip-10-0-136-157.ec2.internal"
Oct 31 21:15:19.359458 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.359440    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/050c66d67c3ed1a5b46b1f7ad6044575-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal\" (UID: \"050c66d67c3ed1a5b46b1f7ad6044575\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal"
Oct 31 21:15:19.359520 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.359466    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6034bd5835de1c87762018f93d657206-config\") pod \"kube-apiserver-proxy-ip-10-0-136-157.ec2.internal\" (UID: \"6034bd5835de1c87762018f93d657206\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-157.ec2.internal"
Oct 31 21:15:19.359520 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.359483    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/050c66d67c3ed1a5b46b1f7ad6044575-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal\" (UID: \"050c66d67c3ed1a5b46b1f7ad6044575\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal"
Oct 31 21:15:19.389643 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:19.389621    2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-157.ec2.internal\" not found"
Oct 31 21:15:19.460614 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.460549    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/050c66d67c3ed1a5b46b1f7ad6044575-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal\" (UID: \"050c66d67c3ed1a5b46b1f7ad6044575\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal"
Oct 31 21:15:19.460614 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.460587    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/050c66d67c3ed1a5b46b1f7ad6044575-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal\" (UID: \"050c66d67c3ed1a5b46b1f7ad6044575\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal"
Oct 31 21:15:19.460614 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.460602    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6034bd5835de1c87762018f93d657206-config\") pod \"kube-apiserver-proxy-ip-10-0-136-157.ec2.internal\" (UID: \"6034bd5835de1c87762018f93d657206\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-157.ec2.internal"
Oct 31 21:15:19.460763 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.460634    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/050c66d67c3ed1a5b46b1f7ad6044575-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal\" (UID: \"050c66d67c3ed1a5b46b1f7ad6044575\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal"
Oct 31 21:15:19.460763 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.460650    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6034bd5835de1c87762018f93d657206-config\") pod \"kube-apiserver-proxy-ip-10-0-136-157.ec2.internal\" (UID: \"6034bd5835de1c87762018f93d657206\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-157.ec2.internal"
Oct 31 21:15:19.460763 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.460637    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/050c66d67c3ed1a5b46b1f7ad6044575-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal\" (UID: \"050c66d67c3ed1a5b46b1f7ad6044575\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal"
Oct 31 21:15:19.490049 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:19.490032    2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-157.ec2.internal\" not found"
Oct 31 21:15:19.590847 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:19.590819    2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-157.ec2.internal\" not found"
Oct 31 21:15:19.636036 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.636016    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal"
Oct 31 21:15:19.638162 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:19.638147    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-157.ec2.internal"
Oct 31 21:15:19.691159 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:19.691026    2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-157.ec2.internal\" not found"
Oct 31 21:15:19.791580 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:19.791509    2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-157.ec2.internal\" not found"
Oct 31 21:15:19.892052 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:19.892022    2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-157.ec2.internal\" not found"
Oct 31 21:15:21.241621 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:21.241596    2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 31 21:15:21.242065 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:21.241766    2575 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 31 21:15:21.249871 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:21.249855    2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-157.ec2.internal\" not found"
Oct 31 21:15:21.352232 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:21.352204    2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-157.ec2.internal\" not found"
Oct 31 21:15:21.430748 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:21.430713    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-30 21:10:20 +0000 UTC" deadline="2028-04-18 09:06:10.407993463 +0000 UTC"
Oct 31 21:15:21.430748 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:21.430741    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12827h50m48.977254742s"
Oct 31 21:15:21.452321 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:21.452301    2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-157.ec2.internal\" not found"
Oct 31 21:15:21.458211 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:21.458183    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod050c66d67c3ed1a5b46b1f7ad6044575.slice/crio-a1a687d9c746d237d1e4e3b121f6060ac051e41b732f5c7792cb1a49c504a9b3 WatchSource:0}: Error finding container a1a687d9c746d237d1e4e3b121f6060ac051e41b732f5c7792cb1a49c504a9b3: Status 404 returned error can't find the container with id a1a687d9c746d237d1e4e3b121f6060ac051e41b732f5c7792cb1a49c504a9b3
Oct 31 21:15:21.458657 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:21.458629    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6034bd5835de1c87762018f93d657206.slice/crio-ba68cb8f841cddeb23c07da3a00ae9d65cf25b8e4a3b2e8ebdb6807d3e6dc8d7 WatchSource:0}: Error finding container ba68cb8f841cddeb23c07da3a00ae9d65cf25b8e4a3b2e8ebdb6807d3e6dc8d7: Status 404 returned error can't find the container with id ba68cb8f841cddeb23c07da3a00ae9d65cf25b8e4a3b2e8ebdb6807d3e6dc8d7
Oct 31 21:15:21.464533 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:21.464514    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 21:15:21.520713 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:21.520691    2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 21:15:21.524477 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:21.524462    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasSufficientMemory"
Oct 31 21:15:21.524541 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:21.524491    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasNoDiskPressure"
Oct 31 21:15:21.524541 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:21.524504    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasSufficientPID"
Oct 31 21:15:21.552866 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:21.552849    2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-157.ec2.internal\" not found"
Oct 31 21:15:21.653385 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:21.653331    2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-157.ec2.internal\" not found"
Oct 31 21:15:21.718495 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:21.718474    2575 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 21:15:21.753886 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:21.753868    2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-157.ec2.internal\" not found"
Oct 31 21:15:21.854546 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:21.854492    2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-157.ec2.internal\" not found"
Oct 31 21:15:21.870952 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:21.870914    2575 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 21:15:21.915249 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:21.915188    2575 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-157.ec2.internal"
Oct 31 21:15:21.940038 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:21.940016    2575 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 21:15:21.942041 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:21.942021    2575 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal"
Oct 31 21:15:21.963874 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:21.963858    2575 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 21:15:22.282024 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.281962    2575 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 21:15:22.286038 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.286010    2575 apiserver.go:52] "Watching apiserver"
Oct 31 21:15:22.313805 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.313781    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 31 21:15:22.318837 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.318803    2575 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 31 21:15:22.319174 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.319152    2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-44v6k","kube-system/konnectivity-agent-4qsqm","openshift-image-registry/node-ca-xhzwc","openshift-network-operator/iptables-alerter-4wdgb","openshift-ovn-kubernetes/ovnkube-node-4ssl9","kube-system/kube-apiserver-proxy-ip-10-0-136-157.ec2.internal","openshift-multus/multus-additional-cni-plugins-ftf4p","openshift-multus/network-metrics-daemon-bwhxr","openshift-dns/node-resolver-wx67m","openshift-network-diagnostics/network-check-target-bsvln","kube-system/global-pull-secret-syncer-9zw4h","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm","openshift-cluster-node-tuning-operator/tuned-d5qv4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal"]
Oct 31 21:15:22.321864 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.321844    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:15:22.325038 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.324485    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.325982 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.325962    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xhzwc"
Oct 31 21:15:22.328114 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.328095    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4wdgb"
Oct 31 21:15:22.330252 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.330236    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.332613 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.332594    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhxr"
Oct 31 21:15:22.332704 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:22.332677    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bwhxr" podUID="3e254918-3cc1-4419-9820-37dff30f4445"
Oct 31 21:15:22.334906 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.334886    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wx67m"
Oct 31 21:15:22.337072 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.337045    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 31 21:15:22.337201 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.337049    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 31 21:15:22.337201 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.337083    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 31 21:15:22.337201 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.337110    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 31 21:15:22.337201 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.337142    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 31 21:15:22.343080 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.341105    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.343080 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.341264    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 31 21:15:22.343080 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.341306    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4qsqm"
Oct 31 21:15:22.343080 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.341315    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 31 21:15:22.343080 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.341375    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xsvd6\""
Oct 31 21:15:22.343080 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.341459    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 31 21:15:22.343080 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.341468    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 31 21:15:22.343080 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.341511    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6mmhq\""
Oct 31 21:15:22.343080 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.341628    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vddbx\""
Oct 31 21:15:22.343080 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.342887    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 31 21:15:22.343080 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.342922    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9gmld\""
Oct 31 21:15:22.343080 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.343017    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 31 21:15:22.343697 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.343153    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 31 21:15:22.343697 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.343262    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 31 21:15:22.345345 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.345111    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-g9h45\""
Oct 31 21:15:22.345345 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.345124    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 31 21:15:22.345345 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.345209    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 31 21:15:22.345345 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.345283    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 31 21:15:22.345345 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.345287    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 31 21:15:22.345345 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.345297    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsvln"
Oct 31 21:15:22.345345 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.345315    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 31 21:15:22.345721 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.345348    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 31 21:15:22.345721 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:22.345373    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsvln" podUID="08d86dfb-548b-4a83-a1c2-d6ce8d4a0091"
Oct 31 21:15:22.347669 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.347649    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:15:22.347812 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:22.347796    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zw4h" podUID="22daf2b3-91ca-462b-8714-0e0b0ebbcb88"
Oct 31 21:15:22.350019 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.350000    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:15:22.362098 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.361987    2575 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 31 21:15:22.416478 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.416454    2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 31 21:15:22.426590 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.426570    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-cni-bin\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.426752 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.426604    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/26069598-5dc4-4c96-b55f-dd2f09884419-ovnkube-script-lib\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.426752 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.426651    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0d542edf-e7a7-4898-bd81-17bd67e18003-host-slash\") pod \"iptables-alerter-4wdgb\" (UID: \"0d542edf-e7a7-4898-bd81-17bd67e18003\") " pod="openshift-network-operator/iptables-alerter-4wdgb"
Oct 31 21:15:22.426752 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.426675    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-log-socket\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.426752 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.426701    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-tuned\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.426752 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.426725    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-systemd-units\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.426752 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.426746    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-var-lib-openvswitch\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.427070 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.426769    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-run-multus-certs\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.427070 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.426794    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-etc-kubernetes\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.427070 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.426817    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-sysctl-d\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.427070 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.426839    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbz7b\" (UniqueName: \"kubernetes.io/projected/5e12af35-fa4d-4c09-b428-a559b5ccc296-kube-api-access-cbz7b\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.427070 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.426864    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e254918-3cc1-4419-9820-37dff30f4445-metrics-certs\") pod \"network-metrics-daemon-bwhxr\" (UID: \"3e254918-3cc1-4419-9820-37dff30f4445\") " pod="openshift-multus/network-metrics-daemon-bwhxr"
Oct 31 21:15:22.427070 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.426888    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-cnibin\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.427070 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.426910    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-host\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.427070 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.426934    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:15:22.427070 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.426977    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-run-ovn\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.427070 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.427011    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/26069598-5dc4-4c96-b55f-dd2f09884419-env-overrides\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.427070 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.427035    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-system-cni-dir\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.427070 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.427059    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-multus-socket-dir-parent\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.427603 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.427143    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-sysconfig\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.427603 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.427177    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-var-lib-kubelet\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.427603 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.427215    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-kubelet-config\") pod \"global-pull-secret-syncer-9zw4h\" (UID: \"22daf2b3-91ca-462b-8714-0e0b0ebbcb88\") " pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:15:22.427603 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.427262    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-device-dir\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:15:22.427603 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.427332    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-sys-fs\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:15:22.427603 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.427372    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-multus-conf-dir\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.427603 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.427407    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-run\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.427603 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.427451    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-cni-binary-copy\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:15:22.427603 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.427498    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:15:22.427603 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.427530    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-slash\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.427603 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.427574    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3d9a2e4e-3a81-4ae6-80ed-68542e996259-konnectivity-ca\") pod \"konnectivity-agent-4qsqm\" (UID: \"3d9a2e4e-3a81-4ae6-80ed-68542e996259\") " pod="kube-system/konnectivity-agent-4qsqm"
Oct 31 21:15:22.428114 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.427614    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e12af35-fa4d-4c09-b428-a559b5ccc296-tmp\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.428114 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.427655    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnrb2\" (UniqueName: \"kubernetes.io/projected/0d542edf-e7a7-4898-bd81-17bd67e18003-kube-api-access-lnrb2\") pod \"iptables-alerter-4wdgb\" (UID: \"0d542edf-e7a7-4898-bd81-17bd67e18003\") " pod="openshift-network-operator/iptables-alerter-4wdgb"
Oct 31 21:15:22.428114 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.427704    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-kubelet\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.428114 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.427743    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjb7n\" (UniqueName: \"kubernetes.io/projected/3e254918-3cc1-4419-9820-37dff30f4445-kube-api-access-mjb7n\") pod \"network-metrics-daemon-bwhxr\" (UID: \"3e254918-3cc1-4419-9820-37dff30f4445\") " pod="openshift-multus/network-metrics-daemon-bwhxr"
Oct 31 21:15:22.428114 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.427798    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-var-lib-cni-multus\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.428114 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.427831    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/91776cb8-cbd4-441a-8d4a-17e342717812-multus-daemon-config\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.428114 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.427857    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-sysctl-conf\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.428114 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.427899    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-original-pull-secret\") pod \"global-pull-secret-syncer-9zw4h\" (UID: \"22daf2b3-91ca-462b-8714-0e0b0ebbcb88\") " pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:15:22.428114 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.427926    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-system-cni-dir\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:15:22.428114 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.428003    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-node-log\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.428114 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.428047    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-var-lib-cni-bin\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.428114 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.428082    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/23da9927-8255-4380-a3f0-b7ea9298ad40-host\") pod \"node-ca-xhzwc\" (UID: \"23da9927-8255-4380-a3f0-b7ea9298ad40\") " pod="openshift-image-registry/node-ca-xhzwc"
Oct 31 21:15:22.428114 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.428107    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-registration-dir\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:15:22.428630 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.428137    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-run-netns\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.428630 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.428162    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-run-ovn-kubernetes\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.428630 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.428223    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfq9g\" (UniqueName: \"kubernetes.io/projected/08d86dfb-548b-4a83-a1c2-d6ce8d4a0091-kube-api-access-pfq9g\") pod \"network-check-target-bsvln\" (UID: \"08d86dfb-548b-4a83-a1c2-d6ce8d4a0091\") " pod="openshift-network-diagnostics/network-check-target-bsvln"
Oct 31 21:15:22.428630 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.428248    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-cni-netd\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.428630 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.428273    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb42ddd8-d246-4bf3-ae35-27aa4584b163-tmp-dir\") pod \"node-resolver-wx67m\" (UID: \"bb42ddd8-d246-4bf3-ae35-27aa4584b163\") " pod="openshift-dns/node-resolver-wx67m"
Oct 31 21:15:22.428630 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.428306    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-multus-cni-dir\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.428630 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.428332    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-run-netns\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.428630 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.428356    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-hostroot\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.428630 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.428387    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-dbus\") pod \"global-pull-secret-syncer-9zw4h\" (UID: \"22daf2b3-91ca-462b-8714-0e0b0ebbcb88\") " pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:15:22.428630 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.428455    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk72s\" (UniqueName: \"kubernetes.io/projected/b65919f8-6cc9-4061-8a67-e1d5455c596d-kube-api-access-pk72s\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:15:22.428630 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.428481    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-cnibin\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:15:22.428630 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.428505    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-etc-openvswitch\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.428630 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.428531    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.428630 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.428560    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-run-k8s-cni-cncf-io\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.428630 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.428601    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0d542edf-e7a7-4898-bd81-17bd67e18003-iptables-alerter-script\") pod \"iptables-alerter-4wdgb\" (UID: \"0d542edf-e7a7-4898-bd81-17bd67e18003\") " pod="openshift-network-operator/iptables-alerter-4wdgb"
Oct 31 21:15:22.428630 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.428633    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-socket-dir\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:15:22.429208 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.428659    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/26069598-5dc4-4c96-b55f-dd2f09884419-ovnkube-config\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.429208 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.428683    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/26069598-5dc4-4c96-b55f-dd2f09884419-ovn-node-metrics-cert\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.429208 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.428710    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-var-lib-kubelet\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.429208 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.428735    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-modprobe-d\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.429208 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.428758    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-sys\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.429208 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.428782    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-etc-selinux\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:15:22.429208 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.428810    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-lib-modules\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.429208 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.428903    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:15:22.429208 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.428949    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c66dd\" (UniqueName: \"kubernetes.io/projected/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-kube-api-access-c66dd\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:15:22.429208 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.428978    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-run-openvswitch\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.429208 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.429006    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktdc9\" (UniqueName: \"kubernetes.io/projected/23da9927-8255-4380-a3f0-b7ea9298ad40-kube-api-access-ktdc9\") pod \"node-ca-xhzwc\" (UID: \"23da9927-8255-4380-a3f0-b7ea9298ad40\") " pod="openshift-image-registry/node-ca-xhzwc"
Oct 31 21:15:22.429208 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.429031    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:15:22.429208 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.429057    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-run-systemd\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.429208 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.429084    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swpfh\" (UniqueName: \"kubernetes.io/projected/26069598-5dc4-4c96-b55f-dd2f09884419-kube-api-access-swpfh\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.429208 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.429110    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bb42ddd8-d246-4bf3-ae35-27aa4584b163-hosts-file\") pod \"node-resolver-wx67m\" (UID: \"bb42ddd8-d246-4bf3-ae35-27aa4584b163\") " pod="openshift-dns/node-resolver-wx67m"
Oct 31 21:15:22.429208 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.429135    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rh6z\" (UniqueName: \"kubernetes.io/projected/bb42ddd8-d246-4bf3-ae35-27aa4584b163-kube-api-access-2rh6z\") pod \"node-resolver-wx67m\" (UID: \"bb42ddd8-d246-4bf3-ae35-27aa4584b163\") " pod="openshift-dns/node-resolver-wx67m"
Oct 31 21:15:22.429798 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.429160    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-systemd\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.429798 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.429187    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91776cb8-cbd4-441a-8d4a-17e342717812-cni-binary-copy\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.429798 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.429222    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb6ph\" (UniqueName: \"kubernetes.io/projected/91776cb8-cbd4-441a-8d4a-17e342717812-kube-api-access-mb6ph\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.429798 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.429248    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3d9a2e4e-3a81-4ae6-80ed-68542e996259-agent-certs\") pod \"konnectivity-agent-4qsqm\" (UID: \"3d9a2e4e-3a81-4ae6-80ed-68542e996259\") " pod="kube-system/konnectivity-agent-4qsqm"
Oct 31 21:15:22.429798 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.429282    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-kubernetes\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.429798 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.429307    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/23da9927-8255-4380-a3f0-b7ea9298ad40-serviceca\") pod \"node-ca-xhzwc\" (UID: \"23da9927-8255-4380-a3f0-b7ea9298ad40\") " pod="openshift-image-registry/node-ca-xhzwc"
Oct 31 21:15:22.438435 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.438416    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 31 21:15:22.438679 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.438662    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 31 21:15:22.438756 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.438716    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 31 21:15:22.438881 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.438865    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 31 21:15:22.438958 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.438903    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 31 21:15:22.439020 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.438959    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 31 21:15:22.439020 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.438985    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 31 21:15:22.439121 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.439028    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-rdjnn\""
Oct 31 21:15:22.439121 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.439040    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-xdqcm\""
Oct 31 21:15:22.439224 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.439148    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 31 21:15:22.439280 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.439236    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ddmwn\""
Oct 31 21:15:22.439536 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.439522    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-p85m4\""
Oct 31 21:15:22.445707 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.445691    2575 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 21:15:22.455217 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.455172    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-157.ec2.internal" event={"ID":"6034bd5835de1c87762018f93d657206","Type":"ContainerStarted","Data":"ba68cb8f841cddeb23c07da3a00ae9d65cf25b8e4a3b2e8ebdb6807d3e6dc8d7"}
Oct 31 21:15:22.456427 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.456399    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal" event={"ID":"050c66d67c3ed1a5b46b1f7ad6044575","Type":"ContainerStarted","Data":"a1a687d9c746d237d1e4e3b121f6060ac051e41b732f5c7792cb1a49c504a9b3"}
Oct 31 21:15:22.460433 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.460415    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-54mtc"
Oct 31 21:15:22.472119 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.472098    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-54mtc"
Oct 31 21:15:22.529564 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.529531    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-var-lib-kubelet\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.529686 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.529579    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-kubelet-config\") pod \"global-pull-secret-syncer-9zw4h\" (UID: \"22daf2b3-91ca-462b-8714-0e0b0ebbcb88\") " pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:15:22.529686 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.529605    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-device-dir\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:15:22.529686 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.529627    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-sys-fs\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:15:22.529686 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.529649    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-multus-conf-dir\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.529686 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.529660    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-var-lib-kubelet\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.529686 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.529662    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-kubelet-config\") pod \"global-pull-secret-syncer-9zw4h\" (UID: \"22daf2b3-91ca-462b-8714-0e0b0ebbcb88\") " pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:15:22.529686 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.529669    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-run\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.530043 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.529710    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-cni-binary-copy\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:15:22.530043 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.529724    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-device-dir\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:15:22.530043 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.529726    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-sys-fs\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:15:22.530043 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.529736    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:15:22.530043 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.529759    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-multus-conf-dir\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.530043 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.529772    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-slash\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.530043 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.529797    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3d9a2e4e-3a81-4ae6-80ed-68542e996259-konnectivity-ca\") pod \"konnectivity-agent-4qsqm\" (UID: \"3d9a2e4e-3a81-4ae6-80ed-68542e996259\") " pod="kube-system/konnectivity-agent-4qsqm"
Oct 31 21:15:22.530043 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.529823    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e12af35-fa4d-4c09-b428-a559b5ccc296-tmp\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.530043 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.529847    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnrb2\" (UniqueName: \"kubernetes.io/projected/0d542edf-e7a7-4898-bd81-17bd67e18003-kube-api-access-lnrb2\") pod \"iptables-alerter-4wdgb\" (UID: \"0d542edf-e7a7-4898-bd81-17bd67e18003\") " pod="openshift-network-operator/iptables-alerter-4wdgb"
Oct 31 21:15:22.530043 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.529871    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-kubelet\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.530043 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.529874    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-run\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.530043 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.529894    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjb7n\" (UniqueName: \"kubernetes.io/projected/3e254918-3cc1-4419-9820-37dff30f4445-kube-api-access-mjb7n\") pod \"network-metrics-daemon-bwhxr\" (UID: \"3e254918-3cc1-4419-9820-37dff30f4445\") " pod="openshift-multus/network-metrics-daemon-bwhxr"
Oct 31 21:15:22.530604 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530159    2575 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 31 21:15:22.530604 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530206    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-var-lib-cni-multus\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.530604 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530246    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/91776cb8-cbd4-441a-8d4a-17e342717812-multus-daemon-config\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.530604 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530277    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-sysctl-conf\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.530604 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530301    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-original-pull-secret\") pod \"global-pull-secret-syncer-9zw4h\" (UID: \"22daf2b3-91ca-462b-8714-0e0b0ebbcb88\") " pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:15:22.530604 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530326    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-system-cni-dir\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:15:22.530604 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530322    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-var-lib-cni-multus\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.530604 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530351    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-node-log\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.530604 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530375    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-var-lib-cni-bin\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.530604 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530385    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-system-cni-dir\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:15:22.530604 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530398    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/23da9927-8255-4380-a3f0-b7ea9298ad40-host\") pod \"node-ca-xhzwc\" (UID: \"23da9927-8255-4380-a3f0-b7ea9298ad40\") " pod="openshift-image-registry/node-ca-xhzwc"
Oct 31 21:15:22.530604 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530421    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-registration-dir\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:15:22.530604 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:22.530429    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 21:15:22.530604 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530436    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-node-log\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.530604 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530443    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-run-netns\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.530604 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530444    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-sysctl-conf\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.530604 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530473    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-run-ovn-kubernetes\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.530604 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530515    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-var-lib-cni-bin\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.531355 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:22.530528    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-original-pull-secret podName:22daf2b3-91ca-462b-8714-0e0b0ebbcb88 nodeName:}" failed. No retries permitted until 2026-10-31 21:15:23.030481105 +0000 UTC m=+3.135482017 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-original-pull-secret") pod "global-pull-secret-syncer-9zw4h" (UID: "22daf2b3-91ca-462b-8714-0e0b0ebbcb88") : object "kube-system"/"original-pull-secret" not registered
Oct 31 21:15:22.531355 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530539    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/23da9927-8255-4380-a3f0-b7ea9298ad40-host\") pod \"node-ca-xhzwc\" (UID: \"23da9927-8255-4380-a3f0-b7ea9298ad40\") " pod="openshift-image-registry/node-ca-xhzwc"
Oct 31 21:15:22.531355 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530540    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-run-ovn-kubernetes\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.531355 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530559    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-slash\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.531355 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530581    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-registration-dir\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:15:22.531355 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530590    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-kubelet\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.531355 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530615    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-run-netns\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.531355 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530647    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfq9g\" (UniqueName: \"kubernetes.io/projected/08d86dfb-548b-4a83-a1c2-d6ce8d4a0091-kube-api-access-pfq9g\") pod \"network-check-target-bsvln\" (UID: \"08d86dfb-548b-4a83-a1c2-d6ce8d4a0091\") " pod="openshift-network-diagnostics/network-check-target-bsvln"
Oct 31 21:15:22.531355 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530674    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-cni-netd\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.531355 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530697    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb42ddd8-d246-4bf3-ae35-27aa4584b163-tmp-dir\") pod \"node-resolver-wx67m\" (UID: \"bb42ddd8-d246-4bf3-ae35-27aa4584b163\") " pod="openshift-dns/node-resolver-wx67m"
Oct 31 21:15:22.531355 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530717    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-multus-cni-dir\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.531355 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530737    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-run-netns\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.531355 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530756    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-hostroot\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.531355 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530778    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-dbus\") pod \"global-pull-secret-syncer-9zw4h\" (UID: \"22daf2b3-91ca-462b-8714-0e0b0ebbcb88\") " pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:15:22.531355 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530799    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk72s\" (UniqueName: \"kubernetes.io/projected/b65919f8-6cc9-4061-8a67-e1d5455c596d-kube-api-access-pk72s\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:15:22.531355 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530821    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-cnibin\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:15:22.531355 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530841    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-etc-openvswitch\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.532181 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530865    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.532181 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530884    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-multus-cni-dir\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.532181 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530889    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-run-k8s-cni-cncf-io\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.532181 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530909    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-run-netns\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.532181 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530912    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0d542edf-e7a7-4898-bd81-17bd67e18003-iptables-alerter-script\") pod \"iptables-alerter-4wdgb\" (UID: \"0d542edf-e7a7-4898-bd81-17bd67e18003\") " pod="openshift-network-operator/iptables-alerter-4wdgb"
Oct 31 21:15:22.532181 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530910    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:15:22.532181 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530933    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-etc-openvswitch\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.532181 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530973    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-cni-netd\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.532181 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530843    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-hostroot\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.532181 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.530999    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-dbus\") pod \"global-pull-secret-syncer-9zw4h\" (UID: \"22daf2b3-91ca-462b-8714-0e0b0ebbcb88\") " pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:15:22.532181 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531010    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-socket-dir\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:15:22.532181 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531037    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/26069598-5dc4-4c96-b55f-dd2f09884419-ovnkube-config\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.532181 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531041    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.532181 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531068    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/26069598-5dc4-4c96-b55f-dd2f09884419-ovn-node-metrics-cert\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.532181 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531068    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb42ddd8-d246-4bf3-ae35-27aa4584b163-tmp-dir\") pod \"node-resolver-wx67m\" (UID: \"bb42ddd8-d246-4bf3-ae35-27aa4584b163\") " pod="openshift-dns/node-resolver-wx67m"
Oct 31 21:15:22.532181 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531096    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-run-k8s-cni-cncf-io\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.532181 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531119    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-var-lib-kubelet\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.533056 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531145    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-modprobe-d\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.533056 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531166    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-sys\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.533056 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531184    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-cnibin\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:15:22.533056 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531188    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-etc-selinux\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:15:22.533056 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531211    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-lib-modules\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.533056 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531233    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:15:22.533056 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531255    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c66dd\" (UniqueName: \"kubernetes.io/projected/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-kube-api-access-c66dd\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:15:22.533056 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531280    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-run-openvswitch\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.533056 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531278    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-socket-dir\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:15:22.533056 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531305    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktdc9\" (UniqueName: \"kubernetes.io/projected/23da9927-8255-4380-a3f0-b7ea9298ad40-kube-api-access-ktdc9\") pod \"node-ca-xhzwc\" (UID: \"23da9927-8255-4380-a3f0-b7ea9298ad40\") " pod="openshift-image-registry/node-ca-xhzwc"
Oct 31 21:15:22.533056 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531331    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:15:22.533056 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531356    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-run-systemd\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.533056 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531378    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-etc-selinux\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:15:22.533056 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531379    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swpfh\" (UniqueName: \"kubernetes.io/projected/26069598-5dc4-4c96-b55f-dd2f09884419-kube-api-access-swpfh\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.533056 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531429    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bb42ddd8-d246-4bf3-ae35-27aa4584b163-hosts-file\") pod \"node-resolver-wx67m\" (UID: \"bb42ddd8-d246-4bf3-ae35-27aa4584b163\") " pod="openshift-dns/node-resolver-wx67m"
Oct 31 21:15:22.533056 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531455    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rh6z\" (UniqueName: \"kubernetes.io/projected/bb42ddd8-d246-4bf3-ae35-27aa4584b163-kube-api-access-2rh6z\") pod \"node-resolver-wx67m\" (UID: \"bb42ddd8-d246-4bf3-ae35-27aa4584b163\") " pod="openshift-dns/node-resolver-wx67m"
Oct 31 21:15:22.533056 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531483    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-systemd\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.533816 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531507    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91776cb8-cbd4-441a-8d4a-17e342717812-cni-binary-copy\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.533816 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531532    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb6ph\" (UniqueName: \"kubernetes.io/projected/91776cb8-cbd4-441a-8d4a-17e342717812-kube-api-access-mb6ph\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.533816 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531570    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3d9a2e4e-3a81-4ae6-80ed-68542e996259-agent-certs\") pod \"konnectivity-agent-4qsqm\" (UID: \"3d9a2e4e-3a81-4ae6-80ed-68542e996259\") " pod="kube-system/konnectivity-agent-4qsqm"
Oct 31 21:15:22.533816 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531595    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-kubernetes\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.533816 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531603    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-lib-modules\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.533816 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531620    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/23da9927-8255-4380-a3f0-b7ea9298ad40-serviceca\") pod \"node-ca-xhzwc\" (UID: \"23da9927-8255-4380-a3f0-b7ea9298ad40\") " pod="openshift-image-registry/node-ca-xhzwc"
Oct 31 21:15:22.533816 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531630    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0d542edf-e7a7-4898-bd81-17bd67e18003-iptables-alerter-script\") pod \"iptables-alerter-4wdgb\" (UID: \"0d542edf-e7a7-4898-bd81-17bd67e18003\") " pod="openshift-network-operator/iptables-alerter-4wdgb"
Oct 31 21:15:22.533816 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531640    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:15:22.533816 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531645    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-cni-bin\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.533816 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531670    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/26069598-5dc4-4c96-b55f-dd2f09884419-ovnkube-script-lib\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.533816 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531695    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-run-systemd\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.533816 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531714    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0d542edf-e7a7-4898-bd81-17bd67e18003-host-slash\") pod \"iptables-alerter-4wdgb\" (UID: \"0d542edf-e7a7-4898-bd81-17bd67e18003\") " pod="openshift-network-operator/iptables-alerter-4wdgb"
Oct 31 21:15:22.533816 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531280    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-cni-binary-copy\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:15:22.533816 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531739    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-log-socket\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.533816 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531779    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-tuned\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.533816 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531802    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-systemd-units\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.533816 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531827    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-var-lib-openvswitch\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.534617 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531865    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-run-multus-certs\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.534617 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531887    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-etc-kubernetes\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.534617 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531912    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-sysctl-d\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.534617 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531920    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-var-lib-kubelet\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.534617 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531951    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbz7b\" (UniqueName: \"kubernetes.io/projected/5e12af35-fa4d-4c09-b428-a559b5ccc296-kube-api-access-cbz7b\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.534617 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531978    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e254918-3cc1-4419-9820-37dff30f4445-metrics-certs\") pod \"network-metrics-daemon-bwhxr\" (UID: \"3e254918-3cc1-4419-9820-37dff30f4445\") " pod="openshift-multus/network-metrics-daemon-bwhxr"
Oct 31 21:15:22.534617 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.532002    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-cnibin\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.534617 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.532011    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-modprobe-d\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.534617 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.532023    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-host\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.534617 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.532079    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-host\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.534617 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531779    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/26069598-5dc4-4c96-b55f-dd2f09884419-ovnkube-config\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.534617 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.532134    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-systemd-units\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.534617 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.532179    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-var-lib-openvswitch\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.534617 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.532219    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-run-multus-certs\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.534617 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.532257    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-etc-kubernetes\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.534617 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531888    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-sys\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.534617 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.532347    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-sysctl-d\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.534617 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.532394    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:15:22.535466 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.532462    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-cni-bin\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.535466 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:22.532586    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 21:15:22.535466 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:22.532635    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e254918-3cc1-4419-9820-37dff30f4445-metrics-certs podName:3e254918-3cc1-4419-9820-37dff30f4445 nodeName:}" failed. No retries permitted until 2026-10-31 21:15:23.0326191 +0000 UTC m=+3.137620005 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3e254918-3cc1-4419-9820-37dff30f4445-metrics-certs") pod "network-metrics-daemon-bwhxr" (UID: "3e254918-3cc1-4419-9820-37dff30f4445") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 21:15:22.535466 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.532688    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-cnibin\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.535466 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.531718    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-run-openvswitch\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.535466 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.533175    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/26069598-5dc4-4c96-b55f-dd2f09884419-ovnkube-script-lib\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.535466 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.533235    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0d542edf-e7a7-4898-bd81-17bd67e18003-host-slash\") pod \"iptables-alerter-4wdgb\" (UID: \"0d542edf-e7a7-4898-bd81-17bd67e18003\") " pod="openshift-network-operator/iptables-alerter-4wdgb"
Oct 31 21:15:22.535466 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.533292    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-kubernetes\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.535466 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.533333    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-log-socket\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.535466 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.533454    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-systemd\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.535466 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.533497    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:15:22.535466 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.533535    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-run-ovn\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.535466 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.533541    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bb42ddd8-d246-4bf3-ae35-27aa4584b163-hosts-file\") pod \"node-resolver-wx67m\" (UID: \"bb42ddd8-d246-4bf3-ae35-27aa4584b163\") " pod="openshift-dns/node-resolver-wx67m"
Oct 31 21:15:22.535466 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.533558    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/26069598-5dc4-4c96-b55f-dd2f09884419-env-overrides\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.535466 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.533585    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-system-cni-dir\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.535466 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.533610    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-multus-socket-dir-parent\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.535466 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.533632    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-sysconfig\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.536171 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.533684    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:15:22.536171 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.533706    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-system-cni-dir\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.536171 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.533721    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-run-ovn\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.536171 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.533745    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-multus-socket-dir-parent\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.536171 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.533962    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/23da9927-8255-4380-a3f0-b7ea9298ad40-serviceca\") pod \"node-ca-xhzwc\" (UID: \"23da9927-8255-4380-a3f0-b7ea9298ad40\") " pod="openshift-image-registry/node-ca-xhzwc"
Oct 31 21:15:22.536171 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.533974    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/26069598-5dc4-4c96-b55f-dd2f09884419-env-overrides\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.536171 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.534159    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-sysconfig\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.536171 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.534270    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/91776cb8-cbd4-441a-8d4a-17e342717812-multus-daemon-config\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.536171 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.534362    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91776cb8-cbd4-441a-8d4a-17e342717812-cni-binary-copy\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.536171 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.534740    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3d9a2e4e-3a81-4ae6-80ed-68542e996259-konnectivity-ca\") pod \"konnectivity-agent-4qsqm\" (UID: \"3d9a2e4e-3a81-4ae6-80ed-68542e996259\") " pod="kube-system/konnectivity-agent-4qsqm"
Oct 31 21:15:22.536171 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.534962    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e12af35-fa4d-4c09-b428-a559b5ccc296-tmp\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.536171 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.535226    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/26069598-5dc4-4c96-b55f-dd2f09884419-ovn-node-metrics-cert\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.536171 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.536012    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-tuned\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.536651 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.536484    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3d9a2e4e-3a81-4ae6-80ed-68542e996259-agent-certs\") pod \"konnectivity-agent-4qsqm\" (UID: \"3d9a2e4e-3a81-4ae6-80ed-68542e996259\") " pod="kube-system/konnectivity-agent-4qsqm"
Oct 31 21:15:22.554364 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:22.554229    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 21:15:22.554364 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:22.554275    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 21:15:22.554364 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:22.554288    2575 projected.go:196] Error preparing data for projected volume kube-api-access-pfq9g for pod openshift-network-diagnostics/network-check-target-bsvln: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 21:15:22.554364 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:22.554342    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/08d86dfb-548b-4a83-a1c2-d6ce8d4a0091-kube-api-access-pfq9g podName:08d86dfb-548b-4a83-a1c2-d6ce8d4a0091 nodeName:}" failed. No retries permitted until 2026-10-31 21:15:23.054325155 +0000 UTC m=+3.159326051 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pfq9g" (UniqueName: "kubernetes.io/projected/08d86dfb-548b-4a83-a1c2-d6ce8d4a0091-kube-api-access-pfq9g") pod "network-check-target-bsvln" (UID: "08d86dfb-548b-4a83-a1c2-d6ce8d4a0091") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 21:15:22.556469 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.556445    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjb7n\" (UniqueName: \"kubernetes.io/projected/3e254918-3cc1-4419-9820-37dff30f4445-kube-api-access-mjb7n\") pod \"network-metrics-daemon-bwhxr\" (UID: \"3e254918-3cc1-4419-9820-37dff30f4445\") " pod="openshift-multus/network-metrics-daemon-bwhxr"
Oct 31 21:15:22.557641 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.557595    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktdc9\" (UniqueName: \"kubernetes.io/projected/23da9927-8255-4380-a3f0-b7ea9298ad40-kube-api-access-ktdc9\") pod \"node-ca-xhzwc\" (UID: \"23da9927-8255-4380-a3f0-b7ea9298ad40\") " pod="openshift-image-registry/node-ca-xhzwc"
Oct 31 21:15:22.557861 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.557844    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk72s\" (UniqueName: \"kubernetes.io/projected/b65919f8-6cc9-4061-8a67-e1d5455c596d-kube-api-access-pk72s\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:15:22.558824 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.558802    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnrb2\" (UniqueName: \"kubernetes.io/projected/0d542edf-e7a7-4898-bd81-17bd67e18003-kube-api-access-lnrb2\") pod \"iptables-alerter-4wdgb\" (UID: \"0d542edf-e7a7-4898-bd81-17bd67e18003\") " pod="openshift-network-operator/iptables-alerter-4wdgb"
Oct 31 21:15:22.559868 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.559845    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rh6z\" (UniqueName: \"kubernetes.io/projected/bb42ddd8-d246-4bf3-ae35-27aa4584b163-kube-api-access-2rh6z\") pod \"node-resolver-wx67m\" (UID: \"bb42ddd8-d246-4bf3-ae35-27aa4584b163\") " pod="openshift-dns/node-resolver-wx67m"
Oct 31 21:15:22.560379 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.560335    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbz7b\" (UniqueName: \"kubernetes.io/projected/5e12af35-fa4d-4c09-b428-a559b5ccc296-kube-api-access-cbz7b\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.560379 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.560370    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb6ph\" (UniqueName: \"kubernetes.io/projected/91776cb8-cbd4-441a-8d4a-17e342717812-kube-api-access-mb6ph\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.561951 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.561925    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c66dd\" (UniqueName: \"kubernetes.io/projected/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-kube-api-access-c66dd\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:15:22.565220 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.565200    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-swpfh\" (UniqueName: \"kubernetes.io/projected/26069598-5dc4-4c96-b55f-dd2f09884419-kube-api-access-swpfh\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.653042 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.653014    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:15:22.655554 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.655524    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:15:22.662293 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.662274    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-44v6k"
Oct 31 21:15:22.668845 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.668824    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xhzwc"
Oct 31 21:15:22.676806 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.676674    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4wdgb"
Oct 31 21:15:22.682817 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:22.682760    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23da9927_8255_4380_a3f0_b7ea9298ad40.slice/crio-a299c657ad84fdbb7a71f68209f2230e4ee9615f37cc5900c4b28c60c8f7f27f WatchSource:0}: Error finding container a299c657ad84fdbb7a71f68209f2230e4ee9615f37cc5900c4b28c60c8f7f27f: Status 404 returned error can't find the container with id a299c657ad84fdbb7a71f68209f2230e4ee9615f37cc5900c4b28c60c8f7f27f
Oct 31 21:15:22.690329 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:22.690115    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e12af35_fa4d_4c09_b428_a559b5ccc296.slice/crio-33afdbfc312b8fa22381d7e5c710b1f51e2ed74ef832769082f7c4770d0c7996 WatchSource:0}: Error finding container 33afdbfc312b8fa22381d7e5c710b1f51e2ed74ef832769082f7c4770d0c7996: Status 404 returned error can't find the container with id 33afdbfc312b8fa22381d7e5c710b1f51e2ed74ef832769082f7c4770d0c7996
Oct 31 21:15:22.692832 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:22.692739    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d542edf_e7a7_4898_bd81_17bd67e18003.slice/crio-18de7a0efc2046a20200fc40d809a3c10c031c1df3bf6a788bd52f6d80cb72b6 WatchSource:0}: Error finding container 18de7a0efc2046a20200fc40d809a3c10c031c1df3bf6a788bd52f6d80cb72b6: Status 404 returned error can't find the container with id 18de7a0efc2046a20200fc40d809a3c10c031c1df3bf6a788bd52f6d80cb72b6
Oct 31 21:15:22.696783 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.696746    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wx67m"
Oct 31 21:15:22.702120 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.702100    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:22.704031 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.704011    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4qsqm"
Oct 31 21:15:22.708034 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:22.707984    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb42ddd8_d246_4bf3_ae35_27aa4584b163.slice/crio-9eae0b17061554df6d278977b921196942e3eee88e7ee69f27ed07a44979b5eb WatchSource:0}: Error finding container 9eae0b17061554df6d278977b921196942e3eee88e7ee69f27ed07a44979b5eb: Status 404 returned error can't find the container with id 9eae0b17061554df6d278977b921196942e3eee88e7ee69f27ed07a44979b5eb
Oct 31 21:15:22.710573 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:22.709929    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:15:22.713167 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:22.713146    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26069598_5dc4_4c96_b55f_dd2f09884419.slice/crio-6d14dcafc9efa958880bd9dd2596c67f2ef87fa4a661a04445f36b4e4ac67382 WatchSource:0}: Error finding container 6d14dcafc9efa958880bd9dd2596c67f2ef87fa4a661a04445f36b4e4ac67382: Status 404 returned error can't find the container with id 6d14dcafc9efa958880bd9dd2596c67f2ef87fa4a661a04445f36b4e4ac67382
Oct 31 21:15:23.037754 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:23.037711    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-original-pull-secret\") pod \"global-pull-secret-syncer-9zw4h\" (UID: \"22daf2b3-91ca-462b-8714-0e0b0ebbcb88\") " pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:15:23.037868 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:23.037804    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e254918-3cc1-4419-9820-37dff30f4445-metrics-certs\") pod \"network-metrics-daemon-bwhxr\" (UID: \"3e254918-3cc1-4419-9820-37dff30f4445\") " pod="openshift-multus/network-metrics-daemon-bwhxr"
Oct 31 21:15:23.037919 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:23.037909    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 21:15:23.037991 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:23.037980    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e254918-3cc1-4419-9820-37dff30f4445-metrics-certs podName:3e254918-3cc1-4419-9820-37dff30f4445 nodeName:}" failed. No retries permitted until 2026-10-31 21:15:24.037962455 +0000 UTC m=+4.142963350 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3e254918-3cc1-4419-9820-37dff30f4445-metrics-certs") pod "network-metrics-daemon-bwhxr" (UID: "3e254918-3cc1-4419-9820-37dff30f4445") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 21:15:23.038038 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:23.038020    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 21:15:23.038090 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:23.038072    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-original-pull-secret podName:22daf2b3-91ca-462b-8714-0e0b0ebbcb88 nodeName:}" failed. No retries permitted until 2026-10-31 21:15:24.038058336 +0000 UTC m=+4.143059240 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-original-pull-secret") pod "global-pull-secret-syncer-9zw4h" (UID: "22daf2b3-91ca-462b-8714-0e0b0ebbcb88") : object "kube-system"/"original-pull-secret" not registered
Oct 31 21:15:23.139186 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:23.139074    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfq9g\" (UniqueName: \"kubernetes.io/projected/08d86dfb-548b-4a83-a1c2-d6ce8d4a0091-kube-api-access-pfq9g\") pod \"network-check-target-bsvln\" (UID: \"08d86dfb-548b-4a83-a1c2-d6ce8d4a0091\") " pod="openshift-network-diagnostics/network-check-target-bsvln"
Oct 31 21:15:23.139186 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:23.139154    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 21:15:23.139186 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:23.139173    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 21:15:23.139186 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:23.139186    2575 projected.go:196] Error preparing data for projected volume kube-api-access-pfq9g for pod openshift-network-diagnostics/network-check-target-bsvln: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 21:15:23.139592 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:23.139241    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/08d86dfb-548b-4a83-a1c2-d6ce8d4a0091-kube-api-access-pfq9g podName:08d86dfb-548b-4a83-a1c2-d6ce8d4a0091 nodeName:}" failed. No retries permitted until 2026-10-31 21:15:24.139223113 +0000 UTC m=+4.244224007 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-pfq9g" (UniqueName: "kubernetes.io/projected/08d86dfb-548b-4a83-a1c2-d6ce8d4a0091-kube-api-access-pfq9g") pod "network-check-target-bsvln" (UID: "08d86dfb-548b-4a83-a1c2-d6ce8d4a0091") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 21:15:23.463670 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:23.462853    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-157.ec2.internal" event={"ID":"6034bd5835de1c87762018f93d657206","Type":"ContainerStarted","Data":"58b2fd0ff9ebe48057cd96a023503fe520281d36acc7b658e84be0732894aa42"}
Oct 31 21:15:23.474049 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:23.473629    2575 generic.go:356] "Generic (PLEG): container finished" podID="050c66d67c3ed1a5b46b1f7ad6044575" containerID="52dab86d69c25d3fe6273ec3333d4be9e53e40aedf149a68ff3ea8917e0fcc99" exitCode=0
Oct 31 21:15:23.474049 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:23.473834    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal" event={"ID":"050c66d67c3ed1a5b46b1f7ad6044575","Type":"ContainerDied","Data":"52dab86d69c25d3fe6273ec3333d4be9e53e40aedf149a68ff3ea8917e0fcc99"}
Oct 31 21:15:23.474894 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:23.474808    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-30 21:10:22 +0000 UTC" deadline="2028-07-12 01:54:55.442839306 +0000 UTC"
Oct 31 21:15:23.474894 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:23.474851    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14860h39m31.967992181s"
Oct 31 21:15:23.488228 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:23.488182    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-d5qv4" event={"ID":"5e12af35-fa4d-4c09-b428-a559b5ccc296","Type":"ContainerStarted","Data":"33afdbfc312b8fa22381d7e5c710b1f51e2ed74ef832769082f7c4770d0c7996"}
Oct 31 21:15:23.495283 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:23.495054    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ftf4p" event={"ID":"94b754a2-a8f3-494c-89bb-e903f9c2cdb0","Type":"ContainerStarted","Data":"351a48ea03ea75f01303bf592add733800f0ce8ce400e0174218d6e3d26cb7e2"}
Oct 31 21:15:23.501676 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:23.501430    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xhzwc" event={"ID":"23da9927-8255-4380-a3f0-b7ea9298ad40","Type":"ContainerStarted","Data":"a299c657ad84fdbb7a71f68209f2230e4ee9615f37cc5900c4b28c60c8f7f27f"}
Oct 31 21:15:23.504447 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:23.504385    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm" event={"ID":"b65919f8-6cc9-4061-8a67-e1d5455c596d","Type":"ContainerStarted","Data":"3cd14d8f549c5518d91e7edb7842878da912dd7fbcb33d5d98cc2328eec1320b"}
Oct 31 21:15:23.505990 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:23.505914    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-157.ec2.internal" podStartSLOduration=2.505898447 podStartE2EDuration="2.505898447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 21:15:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:15:23.504808759 +0000 UTC m=+3.609809669" watchObservedRunningTime="2026-10-31 21:15:23.505898447 +0000 UTC m=+3.610899361"
Oct 31 21:15:23.512769 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:23.512746    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4qsqm" event={"ID":"3d9a2e4e-3a81-4ae6-80ed-68542e996259","Type":"ContainerStarted","Data":"1f17d26e4d3fcaecaf04ec9a71fde34b38a0fef0ad4d08e6d1361dde870d78d4"}
Oct 31 21:15:23.526112 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:23.526080    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9" event={"ID":"26069598-5dc4-4c96-b55f-dd2f09884419","Type":"ContainerStarted","Data":"6d14dcafc9efa958880bd9dd2596c67f2ef87fa4a661a04445f36b4e4ac67382"}
Oct 31 21:15:23.538024 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:23.537973    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wx67m" event={"ID":"bb42ddd8-d246-4bf3-ae35-27aa4584b163","Type":"ContainerStarted","Data":"9eae0b17061554df6d278977b921196942e3eee88e7ee69f27ed07a44979b5eb"}
Oct 31 21:15:23.547244 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:23.547217    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4wdgb" event={"ID":"0d542edf-e7a7-4898-bd81-17bd67e18003","Type":"ContainerStarted","Data":"18de7a0efc2046a20200fc40d809a3c10c031c1df3bf6a788bd52f6d80cb72b6"}
Oct 31 21:15:23.556499 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:23.556473    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-44v6k" event={"ID":"91776cb8-cbd4-441a-8d4a-17e342717812","Type":"ContainerStarted","Data":"95adf62e87edd0f774f4919c151b796072147fad07c03e1766d1343358e0cd1e"}
Oct 31 21:15:24.049391 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:24.048649    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e254918-3cc1-4419-9820-37dff30f4445-metrics-certs\") pod \"network-metrics-daemon-bwhxr\" (UID: \"3e254918-3cc1-4419-9820-37dff30f4445\") " pod="openshift-multus/network-metrics-daemon-bwhxr"
Oct 31 21:15:24.049391 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:24.048700    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-original-pull-secret\") pod \"global-pull-secret-syncer-9zw4h\" (UID: \"22daf2b3-91ca-462b-8714-0e0b0ebbcb88\") " pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:15:24.049391 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:24.048821    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 21:15:24.049391 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:24.048875    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-original-pull-secret podName:22daf2b3-91ca-462b-8714-0e0b0ebbcb88 nodeName:}" failed. No retries permitted until 2026-10-31 21:15:26.048859047 +0000 UTC m=+6.153859953 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-original-pull-secret") pod "global-pull-secret-syncer-9zw4h" (UID: "22daf2b3-91ca-462b-8714-0e0b0ebbcb88") : object "kube-system"/"original-pull-secret" not registered
Oct 31 21:15:24.049391 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:24.049257    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 21:15:24.049391 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:24.049304    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e254918-3cc1-4419-9820-37dff30f4445-metrics-certs podName:3e254918-3cc1-4419-9820-37dff30f4445 nodeName:}" failed. No retries permitted until 2026-10-31 21:15:26.049289853 +0000 UTC m=+6.154290758 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3e254918-3cc1-4419-9820-37dff30f4445-metrics-certs") pod "network-metrics-daemon-bwhxr" (UID: "3e254918-3cc1-4419-9820-37dff30f4445") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 21:15:24.149206 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:24.149171    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfq9g\" (UniqueName: \"kubernetes.io/projected/08d86dfb-548b-4a83-a1c2-d6ce8d4a0091-kube-api-access-pfq9g\") pod \"network-check-target-bsvln\" (UID: \"08d86dfb-548b-4a83-a1c2-d6ce8d4a0091\") " pod="openshift-network-diagnostics/network-check-target-bsvln"
Oct 31 21:15:24.149381 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:24.149332    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 21:15:24.149381 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:24.149351    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 21:15:24.149381 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:24.149362    2575 projected.go:196] Error preparing data for projected volume kube-api-access-pfq9g for pod openshift-network-diagnostics/network-check-target-bsvln: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 21:15:24.149533 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:24.149415    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/08d86dfb-548b-4a83-a1c2-d6ce8d4a0091-kube-api-access-pfq9g podName:08d86dfb-548b-4a83-a1c2-d6ce8d4a0091 nodeName:}" failed. No retries permitted until 2026-10-31 21:15:26.149398778 +0000 UTC m=+6.254399671 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-pfq9g" (UniqueName: "kubernetes.io/projected/08d86dfb-548b-4a83-a1c2-d6ce8d4a0091-kube-api-access-pfq9g") pod "network-check-target-bsvln" (UID: "08d86dfb-548b-4a83-a1c2-d6ce8d4a0091") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 21:15:24.452805 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:24.452396    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsvln"
Oct 31 21:15:24.452805 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:24.452513    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsvln" podUID="08d86dfb-548b-4a83-a1c2-d6ce8d4a0091"
Oct 31 21:15:24.453053 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:24.452886    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:15:24.453053 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:24.452997    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zw4h" podUID="22daf2b3-91ca-462b-8714-0e0b0ebbcb88"
Oct 31 21:15:24.453167 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:24.453059    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhxr"
Oct 31 21:15:24.453167 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:24.453146    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bwhxr" podUID="3e254918-3cc1-4419-9820-37dff30f4445"
Oct 31 21:15:24.475269 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:24.475221    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-30 21:10:22 +0000 UTC" deadline="2028-07-30 22:03:24.184907012 +0000 UTC"
Oct 31 21:15:24.475269 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:24.475257    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15312h47m59.709654896s"
Oct 31 21:15:24.569526 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:24.569489    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal" event={"ID":"050c66d67c3ed1a5b46b1f7ad6044575","Type":"ContainerStarted","Data":"9a5e54d1cd30d9d0e53e74a4d14af2c07bc1481948b07d30ad77ac2591466b1c"}
Oct 31 21:15:24.618380 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:24.618323    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal" podStartSLOduration=3.618298404 podStartE2EDuration="3.618298404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 21:15:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:15:24.609144154 +0000 UTC m=+4.714145070" watchObservedRunningTime="2026-10-31 21:15:24.618298404 +0000 UTC m=+4.723299322"
Oct 31 21:15:25.577808 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:25.577769    2575 generic.go:356] "Generic (PLEG): container finished" podID="94b754a2-a8f3-494c-89bb-e903f9c2cdb0" containerID="2bc21d8c997cbc37ae094c37e1e7e8ccb4879179b97610b353172f235fae11cf" exitCode=0
Oct 31 21:15:25.578394 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:25.577923    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ftf4p" event={"ID":"94b754a2-a8f3-494c-89bb-e903f9c2cdb0","Type":"ContainerDied","Data":"2bc21d8c997cbc37ae094c37e1e7e8ccb4879179b97610b353172f235fae11cf"}
Oct 31 21:15:26.065592 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:26.064786    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-original-pull-secret\") pod \"global-pull-secret-syncer-9zw4h\" (UID: \"22daf2b3-91ca-462b-8714-0e0b0ebbcb88\") " pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:15:26.065592 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:26.064878    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e254918-3cc1-4419-9820-37dff30f4445-metrics-certs\") pod \"network-metrics-daemon-bwhxr\" (UID: \"3e254918-3cc1-4419-9820-37dff30f4445\") " pod="openshift-multus/network-metrics-daemon-bwhxr"
Oct 31 21:15:26.065592 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:26.065023    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 21:15:26.065592 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:26.065087    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e254918-3cc1-4419-9820-37dff30f4445-metrics-certs podName:3e254918-3cc1-4419-9820-37dff30f4445 nodeName:}" failed. No retries permitted until 2026-10-31 21:15:30.065067184 +0000 UTC m=+10.170068093 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3e254918-3cc1-4419-9820-37dff30f4445-metrics-certs") pod "network-metrics-daemon-bwhxr" (UID: "3e254918-3cc1-4419-9820-37dff30f4445") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 21:15:26.065592 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:26.065503    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 21:15:26.065592 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:26.065556    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-original-pull-secret podName:22daf2b3-91ca-462b-8714-0e0b0ebbcb88 nodeName:}" failed. No retries permitted until 2026-10-31 21:15:30.065540742 +0000 UTC m=+10.170541649 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-original-pull-secret") pod "global-pull-secret-syncer-9zw4h" (UID: "22daf2b3-91ca-462b-8714-0e0b0ebbcb88") : object "kube-system"/"original-pull-secret" not registered
Oct 31 21:15:26.166777 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:26.166047    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfq9g\" (UniqueName: \"kubernetes.io/projected/08d86dfb-548b-4a83-a1c2-d6ce8d4a0091-kube-api-access-pfq9g\") pod \"network-check-target-bsvln\" (UID: \"08d86dfb-548b-4a83-a1c2-d6ce8d4a0091\") " pod="openshift-network-diagnostics/network-check-target-bsvln"
Oct 31 21:15:26.166777 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:26.166247    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 21:15:26.166777 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:26.166269    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 21:15:26.166777 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:26.166299    2575 projected.go:196] Error preparing data for projected volume kube-api-access-pfq9g for pod openshift-network-diagnostics/network-check-target-bsvln: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 21:15:26.166777 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:26.166369    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/08d86dfb-548b-4a83-a1c2-d6ce8d4a0091-kube-api-access-pfq9g podName:08d86dfb-548b-4a83-a1c2-d6ce8d4a0091 nodeName:}" failed. No retries permitted until 2026-10-31 21:15:30.166333176 +0000 UTC m=+10.271334079 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-pfq9g" (UniqueName: "kubernetes.io/projected/08d86dfb-548b-4a83-a1c2-d6ce8d4a0091-kube-api-access-pfq9g") pod "network-check-target-bsvln" (UID: "08d86dfb-548b-4a83-a1c2-d6ce8d4a0091") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 21:15:26.452148 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:26.452115    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:15:26.452310 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:26.452124    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsvln"
Oct 31 21:15:26.452310 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:26.452248    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhxr"
Oct 31 21:15:26.452310 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:26.452246    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zw4h" podUID="22daf2b3-91ca-462b-8714-0e0b0ebbcb88"
Oct 31 21:15:26.452465 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:26.452365    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsvln" podUID="08d86dfb-548b-4a83-a1c2-d6ce8d4a0091"
Oct 31 21:15:26.452516 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:26.452473    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bwhxr" podUID="3e254918-3cc1-4419-9820-37dff30f4445"
Oct 31 21:15:27.597641 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:27.597513    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4qsqm" event={"ID":"3d9a2e4e-3a81-4ae6-80ed-68542e996259","Type":"ContainerStarted","Data":"b150a9db194390d50f3fc72345eef144fa83f738755a19822716e73984a07f62"}
Oct 31 21:15:28.450806 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:28.450772    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsvln"
Oct 31 21:15:28.450892 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:28.450772    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhxr"
Oct 31 21:15:28.450964 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:28.450895    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsvln" podUID="08d86dfb-548b-4a83-a1c2-d6ce8d4a0091"
Oct 31 21:15:28.450964 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:28.450785    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:15:28.451075 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:28.451017    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bwhxr" podUID="3e254918-3cc1-4419-9820-37dff30f4445"
Oct 31 21:15:28.451075 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:28.451026    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zw4h" podUID="22daf2b3-91ca-462b-8714-0e0b0ebbcb88"
Oct 31 21:15:28.603214 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:28.603176    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xhzwc" event={"ID":"23da9927-8255-4380-a3f0-b7ea9298ad40","Type":"ContainerStarted","Data":"a8ce1bd6d1454e4bd8eac63aa0c6bb59530d449a8159fb8c89e25fa06849a470"}
Oct 31 21:15:28.605911 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:28.605883    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm" event={"ID":"b65919f8-6cc9-4061-8a67-e1d5455c596d","Type":"ContainerStarted","Data":"58f9eaef387fb7592125c0a6b1231c799de9d6958bd2b971f55682f424c60240"}
Oct 31 21:15:28.641418 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:28.641374    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4qsqm" podStartSLOduration=5.020498918 podStartE2EDuration="8.641358369s" totalImagesPullingTime="3.620859451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:15:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:15:27.640674365 +0000 UTC m=+7.745675277" watchObservedRunningTime="2026-10-31 21:15:28.641358369 +0000 UTC m=+8.746359283"
Oct 31 21:15:28.760295 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:28.760227    2575 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4qsqm"
Oct 31 21:15:28.760810 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:28.760790    2575 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4qsqm"
Oct 31 21:15:28.800667 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:28.800587    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xhzwc" podStartSLOduration=4.071852081 podStartE2EDuration="8.800570285s" totalImagesPullingTime="4.728718204s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 21:15:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:15:28.6421667 +0000 UTC m=+8.747167611" watchObservedRunningTime="2026-10-31 21:15:28.800570285 +0000 UTC m=+8.905571199"
Oct 31 21:15:29.531072 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:29.530923    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4qsqm"
Oct 31 21:15:29.531550 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:29.531534    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4qsqm"
Oct 31 21:15:30.027428 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:30.027405    2575 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 31 21:15:30.101620 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:30.101536    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e254918-3cc1-4419-9820-37dff30f4445-metrics-certs\") pod \"network-metrics-daemon-bwhxr\" (UID: \"3e254918-3cc1-4419-9820-37dff30f4445\") " pod="openshift-multus/network-metrics-daemon-bwhxr"
Oct 31 21:15:30.101620 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:30.101604    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-original-pull-secret\") pod \"global-pull-secret-syncer-9zw4h\" (UID: \"22daf2b3-91ca-462b-8714-0e0b0ebbcb88\") " pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:15:30.101839 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:30.101771    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 21:15:30.101839 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:30.101832    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-original-pull-secret podName:22daf2b3-91ca-462b-8714-0e0b0ebbcb88 nodeName:}" failed. No retries permitted until 2026-10-31 21:15:38.101820158 +0000 UTC m=+18.206821049 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-original-pull-secret") pod "global-pull-secret-syncer-9zw4h" (UID: "22daf2b3-91ca-462b-8714-0e0b0ebbcb88") : object "kube-system"/"original-pull-secret" not registered
Oct 31 21:15:30.102229 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:30.102213    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 21:15:30.102319 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:30.102255    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e254918-3cc1-4419-9820-37dff30f4445-metrics-certs podName:3e254918-3cc1-4419-9820-37dff30f4445 nodeName:}" failed. No retries permitted until 2026-10-31 21:15:38.102243671 +0000 UTC m=+18.207244561 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3e254918-3cc1-4419-9820-37dff30f4445-metrics-certs") pod "network-metrics-daemon-bwhxr" (UID: "3e254918-3cc1-4419-9820-37dff30f4445") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 21:15:30.207126 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:30.202574    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfq9g\" (UniqueName: \"kubernetes.io/projected/08d86dfb-548b-4a83-a1c2-d6ce8d4a0091-kube-api-access-pfq9g\") pod \"network-check-target-bsvln\" (UID: \"08d86dfb-548b-4a83-a1c2-d6ce8d4a0091\") " pod="openshift-network-diagnostics/network-check-target-bsvln"
Oct 31 21:15:30.207126 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:30.202725    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 21:15:30.207126 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:30.202747    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 21:15:30.207126 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:30.202760    2575 projected.go:196] Error preparing data for projected volume kube-api-access-pfq9g for pod openshift-network-diagnostics/network-check-target-bsvln: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 21:15:30.207126 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:30.202820    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/08d86dfb-548b-4a83-a1c2-d6ce8d4a0091-kube-api-access-pfq9g podName:08d86dfb-548b-4a83-a1c2-d6ce8d4a0091 nodeName:}" failed. No retries permitted until 2026-10-31 21:15:38.202802558 +0000 UTC m=+18.307803449 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-pfq9g" (UniqueName: "kubernetes.io/projected/08d86dfb-548b-4a83-a1c2-d6ce8d4a0091-kube-api-access-pfq9g") pod "network-check-target-bsvln" (UID: "08d86dfb-548b-4a83-a1c2-d6ce8d4a0091") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 21:15:30.383865 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:30.382994    2575 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-31T21:15:30.027423422Z","UUID":"c236a9ca-baa2-4fc7-a58e-f7293330274f","Handler":null,"Name":"","Endpoint":""}
Oct 31 21:15:30.386493 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:30.386464    2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 31 21:15:30.386597 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:30.386554    2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 31 21:15:30.452205 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:30.452154    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsvln"
Oct 31 21:15:30.452330 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:30.452288    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsvln" podUID="08d86dfb-548b-4a83-a1c2-d6ce8d4a0091"
Oct 31 21:15:30.452379 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:30.452355    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:15:30.452497 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:30.452476    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zw4h" podUID="22daf2b3-91ca-462b-8714-0e0b0ebbcb88"
Oct 31 21:15:30.452557 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:30.452544    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhxr"
Oct 31 21:15:30.452652 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:30.452636    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bwhxr" podUID="3e254918-3cc1-4419-9820-37dff30f4445"
Oct 31 21:15:30.620495 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:30.619817    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-44v6k" event={"ID":"91776cb8-cbd4-441a-8d4a-17e342717812","Type":"ContainerStarted","Data":"b2ccf4bcaeeece0dcd1fc2eda90ee2d57610d6b7f76fa59ebb965980eb0afc8d"}
Oct 31 21:15:30.627182 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:30.626483    2575 generic.go:356] "Generic (PLEG): container finished" podID="94b754a2-a8f3-494c-89bb-e903f9c2cdb0" containerID="417a0858cb05b0a4cf00fbbf61a0e0501c2e70ae585856b52c0b8963ab69afec" exitCode=0
Oct 31 21:15:30.627182 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:30.626576    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ftf4p" event={"ID":"94b754a2-a8f3-494c-89bb-e903f9c2cdb0","Type":"ContainerDied","Data":"417a0858cb05b0a4cf00fbbf61a0e0501c2e70ae585856b52c0b8963ab69afec"}
Oct 31 21:15:30.632186 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:30.632129    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm" event={"ID":"b65919f8-6cc9-4061-8a67-e1d5455c596d","Type":"ContainerStarted","Data":"54d1a40727fbffd0349f06fb455976a2157b7ee3b95da74088dabdf468655d6f"}
Oct 31 21:15:30.663256 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:30.663205    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-44v6k" podStartSLOduration=3.014296501 podStartE2EDuration="10.663189303s" totalImagesPullingTime="7.648892802s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 21:15:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:15:30.662863476 +0000 UTC m=+10.767864390" watchObservedRunningTime="2026-10-31 21:15:30.663189303 +0000 UTC m=+10.768190217"
Oct 31 21:15:31.636682 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:31.636384    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm" event={"ID":"b65919f8-6cc9-4061-8a67-e1d5455c596d","Type":"ContainerStarted","Data":"d155bec533beb8de4807c8b91dccf1192fac890f1a343a57bba916ba59042158"}
Oct 31 21:15:31.641977 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:31.641907    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wx67m" event={"ID":"bb42ddd8-d246-4bf3-ae35-27aa4584b163","Type":"ContainerStarted","Data":"b619ad37f4c395db33c30b22591adacfa8466417d35dde70c14d0576de7f89c9"}
Oct 31 21:15:31.645031 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:31.645002    2575 generic.go:356] "Generic (PLEG): container finished" podID="94b754a2-a8f3-494c-89bb-e903f9c2cdb0" containerID="9e1b2a5e32243f0fc6f91f1d728d73f95974afb8cde980bab666cf82e12551a2" exitCode=0
Oct 31 21:15:31.645160 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:31.645094    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ftf4p" event={"ID":"94b754a2-a8f3-494c-89bb-e903f9c2cdb0","Type":"ContainerDied","Data":"9e1b2a5e32243f0fc6f91f1d728d73f95974afb8cde980bab666cf82e12551a2"}
Oct 31 21:15:31.680903 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:31.678793    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm" podStartSLOduration=3.825915647 podStartE2EDuration="11.678777638s" totalImagesPullingTime="7.852861991s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 21:15:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:15:31.678383069 +0000 UTC m=+11.783383975" watchObservedRunningTime="2026-10-31 21:15:31.678777638 +0000 UTC m=+11.783778551"
Oct 31 21:15:31.763059 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:31.763005    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-wx67m" podStartSLOduration=3.9417820519999998 podStartE2EDuration="11.762991015s" totalImagesPullingTime="7.821208963s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 21:15:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:15:31.762576214 +0000 UTC m=+11.867577128" watchObservedRunningTime="2026-10-31 21:15:31.762991015 +0000 UTC m=+11.867991928"
Oct 31 21:15:32.450962 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:32.450919    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhxr"
Oct 31 21:15:32.450962 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:32.450955    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsvln"
Oct 31 21:15:32.451168 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:32.451093    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bwhxr" podUID="3e254918-3cc1-4419-9820-37dff30f4445"
Oct 31 21:15:32.451168 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:32.451122    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:15:32.451267 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:32.451208    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsvln" podUID="08d86dfb-548b-4a83-a1c2-d6ce8d4a0091"
Oct 31 21:15:32.451319 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:32.451263    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zw4h" podUID="22daf2b3-91ca-462b-8714-0e0b0ebbcb88"
Oct 31 21:15:32.650296 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:32.650214    2575 generic.go:356] "Generic (PLEG): container finished" podID="94b754a2-a8f3-494c-89bb-e903f9c2cdb0" containerID="156922c32971703eb14808251a2d6138e7db58f1e223b0a6712ac65c34cec560" exitCode=0
Oct 31 21:15:32.650881 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:32.650302    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ftf4p" event={"ID":"94b754a2-a8f3-494c-89bb-e903f9c2cdb0","Type":"ContainerDied","Data":"156922c32971703eb14808251a2d6138e7db58f1e223b0a6712ac65c34cec560"}
Oct 31 21:15:32.651976 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:32.651955    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4wdgb" event={"ID":"0d542edf-e7a7-4898-bd81-17bd67e18003","Type":"ContainerStarted","Data":"4385cd739b0cf0b6c737886d4905613dc71120b7e6b0ece5b4579c6f997189ea"}
Oct 31 21:15:32.653875 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:32.653851    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-d5qv4" event={"ID":"5e12af35-fa4d-4c09-b428-a559b5ccc296","Type":"ContainerStarted","Data":"57d4aa89f50c1b4f2ac8101621b48af2ef8cfc4a2b135eb251114692216118c9"}
Oct 31 21:15:32.743462 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:32.743411    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-d5qv4" podStartSLOduration=3.209205916 podStartE2EDuration="12.743396988s" totalImagesPullingTime="9.534191072s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 21:15:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:15:32.743203733 +0000 UTC m=+12.848204646" watchObservedRunningTime="2026-10-31 21:15:32.743396988 +0000 UTC m=+12.848397903"
Oct 31 21:15:32.782059 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:32.782018    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4wdgb" podStartSLOduration=5.137142538 podStartE2EDuration="12.782003191s" totalImagesPullingTime="7.644860653s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 21:15:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:15:32.781881081 +0000 UTC m=+12.886881992" watchObservedRunningTime="2026-10-31 21:15:32.782003191 +0000 UTC m=+12.887004158"
Oct 31 21:15:34.451100 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:34.451069    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsvln"
Oct 31 21:15:34.451746 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:34.451126    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhxr"
Oct 31 21:15:34.451746 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:34.451136    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:15:34.451746 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:34.451229    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsvln" podUID="08d86dfb-548b-4a83-a1c2-d6ce8d4a0091"
Oct 31 21:15:34.451746 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:34.451353    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bwhxr" podUID="3e254918-3cc1-4419-9820-37dff30f4445"
Oct 31 21:15:34.451746 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:34.451431    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zw4h" podUID="22daf2b3-91ca-462b-8714-0e0b0ebbcb88"
Oct 31 21:15:34.661456 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:34.661369    2575 generic.go:356] "Generic (PLEG): container finished" podID="94b754a2-a8f3-494c-89bb-e903f9c2cdb0" containerID="3fd1811be54ec6d2a66b3ec47973cacf08b0987bcdb4f42a6405fba6dbcfbd09" exitCode=0
Oct 31 21:15:34.661456 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:34.661431    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ftf4p" event={"ID":"94b754a2-a8f3-494c-89bb-e903f9c2cdb0","Type":"ContainerDied","Data":"3fd1811be54ec6d2a66b3ec47973cacf08b0987bcdb4f42a6405fba6dbcfbd09"}
Oct 31 21:15:35.667151 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:35.666906    2575 generic.go:356] "Generic (PLEG): container finished" podID="94b754a2-a8f3-494c-89bb-e903f9c2cdb0" containerID="4045d9f0e5611a94da16771cb969f6c7634f11d0c51f7b4c580f234db6d6025b" exitCode=0
Oct 31 21:15:35.667610 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:35.667174    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ftf4p" event={"ID":"94b754a2-a8f3-494c-89bb-e903f9c2cdb0","Type":"ContainerDied","Data":"4045d9f0e5611a94da16771cb969f6c7634f11d0c51f7b4c580f234db6d6025b"}
Oct 31 21:15:36.450013 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:36.449982    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsvln"
Oct 31 21:15:36.450013 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:36.450001    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhxr"
Oct 31 21:15:36.450219 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:36.449987    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:15:36.450219 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:36.450082    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsvln" podUID="08d86dfb-548b-4a83-a1c2-d6ce8d4a0091"
Oct 31 21:15:36.450219 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:36.450190    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bwhxr" podUID="3e254918-3cc1-4419-9820-37dff30f4445"
Oct 31 21:15:36.450483 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:36.450277    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zw4h" podUID="22daf2b3-91ca-462b-8714-0e0b0ebbcb88"
Oct 31 21:15:36.671749 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:36.671720    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ftf4p" event={"ID":"94b754a2-a8f3-494c-89bb-e903f9c2cdb0","Type":"ContainerStarted","Data":"d6fdebaba459f5137f4bd95f0bb31027054a0d9d910a57fa468e18b21da98094"}
Oct 31 21:15:36.743837 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:36.743739    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ftf4p" podStartSLOduration=6.878617523 podStartE2EDuration="16.743726108s" totalImagesPullingTime="9.865108585s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 21:15:20 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:15:36.743140423 +0000 UTC m=+16.848141335" watchObservedRunningTime="2026-10-31 21:15:36.743726108 +0000 UTC m=+16.848727019"
Oct 31 21:15:37.676201 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:37.676056    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ssl9_26069598-5dc4-4c96-b55f-dd2f09884419/ovn-acl-logging/0.log"
Oct 31 21:15:37.676566 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:37.676470    2575 generic.go:356] "Generic (PLEG): container finished" podID="26069598-5dc4-4c96-b55f-dd2f09884419" containerID="f195879d5c7baf5c73f623c2c35682fb10da49088b7582a3fc72f326b9a8f67a" exitCode=1
Oct 31 21:15:37.676566 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:37.676537    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9" event={"ID":"26069598-5dc4-4c96-b55f-dd2f09884419","Type":"ContainerStarted","Data":"cfa6f06c4302f3f85a2d351287856c5e62010db516028c54413fe28f57b89dfc"}
Oct 31 21:15:37.676629 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:37.676603    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9" event={"ID":"26069598-5dc4-4c96-b55f-dd2f09884419","Type":"ContainerStarted","Data":"bdb66120c7957220a43acd5a64f1fc54d30a93fde3f315f0a7d7fda95e1fde18"}
Oct 31 21:15:37.676659 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:37.676644    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9" event={"ID":"26069598-5dc4-4c96-b55f-dd2f09884419","Type":"ContainerStarted","Data":"5e51c9214906809d2ff9c2f486237324692f07f4b61b5fc50ab719eca6512834"}
Oct 31 21:15:37.676701 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:37.676689    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9" event={"ID":"26069598-5dc4-4c96-b55f-dd2f09884419","Type":"ContainerStarted","Data":"4482eed38c5955a41338b97f9d2e05cbf9bf219514b221d84b1e531fff0ed861"}
Oct 31 21:15:37.676772 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:37.676737    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9" event={"ID":"26069598-5dc4-4c96-b55f-dd2f09884419","Type":"ContainerDied","Data":"f195879d5c7baf5c73f623c2c35682fb10da49088b7582a3fc72f326b9a8f67a"}
Oct 31 21:15:37.676803 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:37.676785    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9" event={"ID":"26069598-5dc4-4c96-b55f-dd2f09884419","Type":"ContainerStarted","Data":"7b0f8a271173c5a9b2e4c639402bd1129ddba9a97c501fe1c6f881b76d5c54d2"}
Oct 31 21:15:38.167931 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:38.167859    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e254918-3cc1-4419-9820-37dff30f4445-metrics-certs\") pod \"network-metrics-daemon-bwhxr\" (UID: \"3e254918-3cc1-4419-9820-37dff30f4445\") " pod="openshift-multus/network-metrics-daemon-bwhxr"
Oct 31 21:15:38.167931 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:38.167903    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-original-pull-secret\") pod \"global-pull-secret-syncer-9zw4h\" (UID: \"22daf2b3-91ca-462b-8714-0e0b0ebbcb88\") " pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:15:38.168134 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:38.168013    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 21:15:38.168134 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:38.168022    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 21:15:38.168134 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:38.168067    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e254918-3cc1-4419-9820-37dff30f4445-metrics-certs podName:3e254918-3cc1-4419-9820-37dff30f4445 nodeName:}" failed. No retries permitted until 2026-10-31 21:15:54.168053951 +0000 UTC m=+34.273054854 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3e254918-3cc1-4419-9820-37dff30f4445-metrics-certs") pod "network-metrics-daemon-bwhxr" (UID: "3e254918-3cc1-4419-9820-37dff30f4445") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 21:15:38.168134 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:38.168080    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-original-pull-secret podName:22daf2b3-91ca-462b-8714-0e0b0ebbcb88 nodeName:}" failed. No retries permitted until 2026-10-31 21:15:54.168074789 +0000 UTC m=+34.273075679 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-original-pull-secret") pod "global-pull-secret-syncer-9zw4h" (UID: "22daf2b3-91ca-462b-8714-0e0b0ebbcb88") : object "kube-system"/"original-pull-secret" not registered
Oct 31 21:15:38.269122 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:38.269094    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfq9g\" (UniqueName: \"kubernetes.io/projected/08d86dfb-548b-4a83-a1c2-d6ce8d4a0091-kube-api-access-pfq9g\") pod \"network-check-target-bsvln\" (UID: \"08d86dfb-548b-4a83-a1c2-d6ce8d4a0091\") " pod="openshift-network-diagnostics/network-check-target-bsvln"
Oct 31 21:15:38.269252 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:38.269218    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 21:15:38.269252 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:38.269233    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 21:15:38.269252 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:38.269241    2575 projected.go:196] Error preparing data for projected volume kube-api-access-pfq9g for pod openshift-network-diagnostics/network-check-target-bsvln: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 21:15:38.269347 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:38.269279    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/08d86dfb-548b-4a83-a1c2-d6ce8d4a0091-kube-api-access-pfq9g podName:08d86dfb-548b-4a83-a1c2-d6ce8d4a0091 nodeName:}" failed. No retries permitted until 2026-10-31 21:15:54.269267793 +0000 UTC m=+34.374268683 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-pfq9g" (UniqueName: "kubernetes.io/projected/08d86dfb-548b-4a83-a1c2-d6ce8d4a0091-kube-api-access-pfq9g") pod "network-check-target-bsvln" (UID: "08d86dfb-548b-4a83-a1c2-d6ce8d4a0091") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 21:15:38.450494 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:38.450419    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:15:38.450645 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:38.450535    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhxr"
Oct 31 21:15:38.450645 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:38.450541    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zw4h" podUID="22daf2b3-91ca-462b-8714-0e0b0ebbcb88"
Oct 31 21:15:38.450645 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:38.450631    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bwhxr" podUID="3e254918-3cc1-4419-9820-37dff30f4445"
Oct 31 21:15:38.450811 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:38.450675    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsvln"
Oct 31 21:15:38.450811 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:38.450730    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsvln" podUID="08d86dfb-548b-4a83-a1c2-d6ce8d4a0091"
Oct 31 21:15:39.683671 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:39.683604    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ssl9_26069598-5dc4-4c96-b55f-dd2f09884419/ovn-acl-logging/0.log"
Oct 31 21:15:39.684138 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:39.683959    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9" event={"ID":"26069598-5dc4-4c96-b55f-dd2f09884419","Type":"ContainerStarted","Data":"11c441f9f8540dfd25aa3ffeb4e0c2a9d2938c791831835f7e8d8557ccd0f442"}
Oct 31 21:15:40.451078 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:40.451046    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhxr"
Oct 31 21:15:40.451235 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:40.451201    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:15:40.451283 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:40.451222    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bwhxr" podUID="3e254918-3cc1-4419-9820-37dff30f4445"
Oct 31 21:15:40.451332 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:40.451260    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zw4h" podUID="22daf2b3-91ca-462b-8714-0e0b0ebbcb88"
Oct 31 21:15:40.451332 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:40.451302    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsvln"
Oct 31 21:15:40.451396 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:40.451372    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsvln" podUID="08d86dfb-548b-4a83-a1c2-d6ce8d4a0091"
Oct 31 21:15:41.692298 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:41.692232    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ssl9_26069598-5dc4-4c96-b55f-dd2f09884419/ovn-acl-logging/0.log"
Oct 31 21:15:41.693057 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:41.692544    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9" event={"ID":"26069598-5dc4-4c96-b55f-dd2f09884419","Type":"ContainerStarted","Data":"d1ccff96df17595b65fa5a309e468e85caf50c1729e8c48b532dfa26f4468791"}
Oct 31 21:15:41.693057 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:41.693041    2575 scope.go:122] "RemoveContainer" containerID="f195879d5c7baf5c73f623c2c35682fb10da49088b7582a3fc72f326b9a8f67a"
Oct 31 21:15:41.693169 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:41.693040    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:41.693221 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:41.693191    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:41.708640 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:41.708460    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:41.708778 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:41.708765    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:41.992414 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:41.992311    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:42.161666 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:42.161622    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:42.450730 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:42.450697    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:15:42.450730 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:42.450724    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsvln"
Oct 31 21:15:42.450985 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:42.450796    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zw4h" podUID="22daf2b3-91ca-462b-8714-0e0b0ebbcb88"
Oct 31 21:15:42.450985 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:42.450894    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsvln" podUID="08d86dfb-548b-4a83-a1c2-d6ce8d4a0091"
Oct 31 21:15:42.450985 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:42.450965    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhxr"
Oct 31 21:15:42.451148 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:42.451035    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bwhxr" podUID="3e254918-3cc1-4419-9820-37dff30f4445"
Oct 31 21:15:42.698418 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:42.698392    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ssl9_26069598-5dc4-4c96-b55f-dd2f09884419/ovn-acl-logging/0.log"
Oct 31 21:15:42.698871 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:42.698843    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9" event={"ID":"26069598-5dc4-4c96-b55f-dd2f09884419","Type":"ContainerStarted","Data":"744f1bf61decc6ad0f194f39de4307042c09efad04079a15f89c8024ed28d156"}
Oct 31 21:15:42.699003 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:42.698984    2575 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 31 21:15:42.771606 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:42.771347    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9" podStartSLOduration=8.53535072 podStartE2EDuration="22.77133127s" totalImagesPullingTime="14.23598055s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 21:15:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:15:42.770339508 +0000 UTC m=+22.875340419" watchObservedRunningTime="2026-10-31 21:15:42.77133127 +0000 UTC m=+22.876332184"
Oct 31 21:15:43.446693 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:43.446629    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9zw4h"]
Oct 31 21:15:43.446842 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:43.446728    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:15:43.446842 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:43.446811    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zw4h" podUID="22daf2b3-91ca-462b-8714-0e0b0ebbcb88"
Oct 31 21:15:43.450138 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:43.450108    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bwhxr"]
Oct 31 21:15:43.450323 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:43.450308    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhxr"
Oct 31 21:15:43.450423 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:43.450402    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bwhxr" podUID="3e254918-3cc1-4419-9820-37dff30f4445"
Oct 31 21:15:43.450876 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:43.450853    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bsvln"]
Oct 31 21:15:43.450973 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:43.450924    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsvln"
Oct 31 21:15:43.451028 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:43.451007    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsvln" podUID="08d86dfb-548b-4a83-a1c2-d6ce8d4a0091"
Oct 31 21:15:43.701231 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:43.701162    2575 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 31 21:15:45.171414 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:45.171373    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:15:45.450682 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:45.450610    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhxr"
Oct 31 21:15:45.450682 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:45.450652    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:15:45.450844 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:45.450731    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bwhxr" podUID="3e254918-3cc1-4419-9820-37dff30f4445"
Oct 31 21:15:45.450844 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:45.450813    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsvln"
Oct 31 21:15:45.451007 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:45.450917    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsvln" podUID="08d86dfb-548b-4a83-a1c2-d6ce8d4a0091"
Oct 31 21:15:45.451064 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:45.451038    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zw4h" podUID="22daf2b3-91ca-462b-8714-0e0b0ebbcb88"
Oct 31 21:15:46.541996 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:46.541965    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:46.651790 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:46.651681    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:47.450378 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:47.450347    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhxr"
Oct 31 21:15:47.450378 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:47.450377    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsvln"
Oct 31 21:15:47.450583 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:47.450348    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:15:47.450583 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:47.450486    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bwhxr" podUID="3e254918-3cc1-4419-9820-37dff30f4445"
Oct 31 21:15:47.450654 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:47.450588    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zw4h" podUID="22daf2b3-91ca-462b-8714-0e0b0ebbcb88"
Oct 31 21:15:47.450687 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:47.450650    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bsvln" podUID="08d86dfb-548b-4a83-a1c2-d6ce8d4a0091"
Oct 31 21:15:48.268297 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:48.268266    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:48.374594 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:48.374564    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:48.763894 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:48.763870    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:48.878640 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:48.878583    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:48.988438 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:48.988375    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:49.100448 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.100415    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:49.232152 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.232134    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeReady"
Oct 31 21:15:49.232250 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.232220    2575 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 31 21:15:49.287830 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.287808    2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-x8cxm"]
Oct 31 21:15:49.291012 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.290991    2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-47zb8"]
Oct 31 21:15:49.291012 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.291012    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-x8cxm"
Oct 31 21:15:49.293927 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.293905    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:15:49.309070 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.309047    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 31 21:15:49.309159 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.309074    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 31 21:15:49.309911 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.309891    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-x8cxm"]
Oct 31 21:15:49.310096 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.310082    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-nq8jq\""
Oct 31 21:15:49.310096 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.310089    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 31 21:15:49.311541 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.311520    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 31 21:15:49.322320 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.322304    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-47zb8"]
Oct 31 21:15:49.322888 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.322868    2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-fl2tq"]
Oct 31 21:15:49.326008 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.325994    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fl2tq"
Oct 31 21:15:49.344993 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.344969    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fl2tq"]
Oct 31 21:15:49.413614 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.413451    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 31 21:15:49.413614 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.413522    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 31 21:15:49.413614 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.413531    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 31 21:15:49.413806 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.413531    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 31 21:15:49.413806 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.413559    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dwlts\""
Oct 31 21:15:49.413806 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.413581    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-866d2\""
Oct 31 21:15:49.413806 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.413600    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 31 21:15:49.451078 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.451051    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsvln"
Oct 31 21:15:49.451205 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.451163    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhxr"
Oct 31 21:15:49.451279 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.451264    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:15:49.451637 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.451596    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pknd\" (UniqueName: \"kubernetes.io/projected/1695694b-12a1-4a6c-b931-bf7dda9ec50e-kube-api-access-9pknd\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:15:49.451637 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.451634    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6207663-1cfa-48d9-8382-12973bbf29fd-metrics-tls\") pod \"dns-default-fl2tq\" (UID: \"a6207663-1cfa-48d9-8382-12973bbf29fd\") " pod="openshift-dns/dns-default-fl2tq"
Oct 31 21:15:49.451823 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.451722    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97qrk\" (UniqueName: \"kubernetes.io/projected/6d63e78f-f474-4966-adb5-401be54f4ab4-kube-api-access-97qrk\") pod \"ingress-canary-x8cxm\" (UID: \"6d63e78f-f474-4966-adb5-401be54f4ab4\") " pod="openshift-ingress-canary/ingress-canary-x8cxm"
Oct 31 21:15:49.451823 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.451802    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a6207663-1cfa-48d9-8382-12973bbf29fd-config-volume\") pod \"dns-default-fl2tq\" (UID: \"a6207663-1cfa-48d9-8382-12973bbf29fd\") " pod="openshift-dns/dns-default-fl2tq"
Oct 31 21:15:49.451932 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.451835    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnxs9\" (UniqueName: \"kubernetes.io/projected/a6207663-1cfa-48d9-8382-12973bbf29fd-kube-api-access-mnxs9\") pod \"dns-default-fl2tq\" (UID: \"a6207663-1cfa-48d9-8382-12973bbf29fd\") " pod="openshift-dns/dns-default-fl2tq"
Oct 31 21:15:49.451932 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.451866    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1695694b-12a1-4a6c-b931-bf7dda9ec50e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:15:49.452056 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.451964    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6207663-1cfa-48d9-8382-12973bbf29fd-tmp-dir\") pod \"dns-default-fl2tq\" (UID: \"a6207663-1cfa-48d9-8382-12973bbf29fd\") " pod="openshift-dns/dns-default-fl2tq"
Oct 31 21:15:49.452056 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.451989    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1695694b-12a1-4a6c-b931-bf7dda9ec50e-crio-socket\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:15:49.452056 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.452004    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1695694b-12a1-4a6c-b931-bf7dda9ec50e-data-volume\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:15:49.452056 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.452022    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1695694b-12a1-4a6c-b931-bf7dda9ec50e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:15:49.452245 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.452071    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d63e78f-f474-4966-adb5-401be54f4ab4-cert\") pod \"ingress-canary-x8cxm\" (UID: \"6d63e78f-f474-4966-adb5-401be54f4ab4\") " pod="openshift-ingress-canary/ingress-canary-x8cxm"
Oct 31 21:15:49.509330 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.509308    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 31 21:15:49.509440 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.509400    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 31 21:15:49.509489 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.509438    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 31 21:15:49.509522 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.509506    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 31 21:15:49.509558 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.509538    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cvv8l\""
Oct 31 21:15:49.509558 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.509539    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-sflcb\""
Oct 31 21:15:49.509620 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.509562    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-j68t2\""
Oct 31 21:15:49.515577 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.515558    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:49.552676 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.552626    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnxs9\" (UniqueName: \"kubernetes.io/projected/a6207663-1cfa-48d9-8382-12973bbf29fd-kube-api-access-mnxs9\") pod \"dns-default-fl2tq\" (UID: \"a6207663-1cfa-48d9-8382-12973bbf29fd\") " pod="openshift-dns/dns-default-fl2tq"
Oct 31 21:15:49.552676 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.552665    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1695694b-12a1-4a6c-b931-bf7dda9ec50e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:15:49.552798 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.552694    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6207663-1cfa-48d9-8382-12973bbf29fd-tmp-dir\") pod \"dns-default-fl2tq\" (UID: \"a6207663-1cfa-48d9-8382-12973bbf29fd\") " pod="openshift-dns/dns-default-fl2tq"
Oct 31 21:15:49.552798 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.552723    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1695694b-12a1-4a6c-b931-bf7dda9ec50e-crio-socket\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:15:49.552798 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.552747    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1695694b-12a1-4a6c-b931-bf7dda9ec50e-data-volume\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:15:49.552798 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.552771    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1695694b-12a1-4a6c-b931-bf7dda9ec50e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:15:49.552798 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:49.552791    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 21:15:49.552798 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.552796    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d63e78f-f474-4966-adb5-401be54f4ab4-cert\") pod \"ingress-canary-x8cxm\" (UID: \"6d63e78f-f474-4966-adb5-401be54f4ab4\") " pod="openshift-ingress-canary/ingress-canary-x8cxm"
Oct 31 21:15:49.553084 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:49.552850    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1695694b-12a1-4a6c-b931-bf7dda9ec50e-insights-runtime-extractor-tls podName:1695694b-12a1-4a6c-b931-bf7dda9ec50e nodeName:}" failed. No retries permitted until 2026-10-31 21:15:50.052833198 +0000 UTC m=+30.157834088 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1695694b-12a1-4a6c-b931-bf7dda9ec50e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-47zb8" (UID: "1695694b-12a1-4a6c-b931-bf7dda9ec50e") : secret "insights-runtime-extractor-tls" not found
Oct 31 21:15:49.553084 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.552869    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pknd\" (UniqueName: \"kubernetes.io/projected/1695694b-12a1-4a6c-b931-bf7dda9ec50e-kube-api-access-9pknd\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:15:49.553084 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:49.552872    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 31 21:15:49.553084 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.552901    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6207663-1cfa-48d9-8382-12973bbf29fd-metrics-tls\") pod \"dns-default-fl2tq\" (UID: \"a6207663-1cfa-48d9-8382-12973bbf29fd\") " pod="openshift-dns/dns-default-fl2tq"
Oct 31 21:15:49.553084 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:49.552950    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1695694b-12a1-4a6c-b931-bf7dda9ec50e-kube-rbac-proxy-cm podName:1695694b-12a1-4a6c-b931-bf7dda9ec50e nodeName:}" failed. No retries permitted until 2026-10-31 21:15:50.05291939 +0000 UTC m=+30.157920293 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1695694b-12a1-4a6c-b931-bf7dda9ec50e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-47zb8" (UID: "1695694b-12a1-4a6c-b931-bf7dda9ec50e") : configmap "kube-rbac-proxy" not found
Oct 31 21:15:49.553084 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:49.552986    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 21:15:49.553084 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.552996    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97qrk\" (UniqueName: \"kubernetes.io/projected/6d63e78f-f474-4966-adb5-401be54f4ab4-kube-api-access-97qrk\") pod \"ingress-canary-x8cxm\" (UID: \"6d63e78f-f474-4966-adb5-401be54f4ab4\") " pod="openshift-ingress-canary/ingress-canary-x8cxm"
Oct 31 21:15:49.553084 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:49.553027    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6207663-1cfa-48d9-8382-12973bbf29fd-metrics-tls podName:a6207663-1cfa-48d9-8382-12973bbf29fd nodeName:}" failed. No retries permitted until 2026-10-31 21:15:50.053012066 +0000 UTC m=+30.158012959 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6207663-1cfa-48d9-8382-12973bbf29fd-metrics-tls") pod "dns-default-fl2tq" (UID: "a6207663-1cfa-48d9-8382-12973bbf29fd") : secret "dns-default-metrics-tls" not found
Oct 31 21:15:49.553084 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.553058    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a6207663-1cfa-48d9-8382-12973bbf29fd-config-volume\") pod \"dns-default-fl2tq\" (UID: \"a6207663-1cfa-48d9-8382-12973bbf29fd\") " pod="openshift-dns/dns-default-fl2tq"
Oct 31 21:15:49.553476 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.553114    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1695694b-12a1-4a6c-b931-bf7dda9ec50e-crio-socket\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:15:49.553476 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.553322    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1695694b-12a1-4a6c-b931-bf7dda9ec50e-data-volume\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:15:49.553476 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.553325    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6207663-1cfa-48d9-8382-12973bbf29fd-tmp-dir\") pod \"dns-default-fl2tq\" (UID: \"a6207663-1cfa-48d9-8382-12973bbf29fd\") " pod="openshift-dns/dns-default-fl2tq"
Oct 31 21:15:49.553610 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.553567    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a6207663-1cfa-48d9-8382-12973bbf29fd-config-volume\") pod \"dns-default-fl2tq\" (UID: \"a6207663-1cfa-48d9-8382-12973bbf29fd\") " pod="openshift-dns/dns-default-fl2tq"
Oct 31 21:15:49.556586 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.556567    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d63e78f-f474-4966-adb5-401be54f4ab4-cert\") pod \"ingress-canary-x8cxm\" (UID: \"6d63e78f-f474-4966-adb5-401be54f4ab4\") " pod="openshift-ingress-canary/ingress-canary-x8cxm"
Oct 31 21:15:49.577534 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.577511    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnxs9\" (UniqueName: \"kubernetes.io/projected/a6207663-1cfa-48d9-8382-12973bbf29fd-kube-api-access-mnxs9\") pod \"dns-default-fl2tq\" (UID: \"a6207663-1cfa-48d9-8382-12973bbf29fd\") " pod="openshift-dns/dns-default-fl2tq"
Oct 31 21:15:49.577853 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.577833    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pknd\" (UniqueName: \"kubernetes.io/projected/1695694b-12a1-4a6c-b931-bf7dda9ec50e-kube-api-access-9pknd\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:15:49.578426 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.578405    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97qrk\" (UniqueName: \"kubernetes.io/projected/6d63e78f-f474-4966-adb5-401be54f4ab4-kube-api-access-97qrk\") pod \"ingress-canary-x8cxm\" (UID: \"6d63e78f-f474-4966-adb5-401be54f4ab4\") " pod="openshift-ingress-canary/ingress-canary-x8cxm"
Oct 31 21:15:49.603743 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.603724    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-x8cxm"
Oct 31 21:15:49.719403 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.719370    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:49.905768 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:49.905712    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-x8cxm"]
Oct 31 21:15:49.910609 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:49.910583    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d63e78f_f474_4966_adb5_401be54f4ab4.slice/crio-2b851e26be51e365888088ee65bfb2f98feba4569e308ae07cc00b5b172091c4 WatchSource:0}: Error finding container 2b851e26be51e365888088ee65bfb2f98feba4569e308ae07cc00b5b172091c4: Status 404 returned error can't find the container with id 2b851e26be51e365888088ee65bfb2f98feba4569e308ae07cc00b5b172091c4
Oct 31 21:15:50.051928 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:50.051898    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:50.056906 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:50.056886    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1695694b-12a1-4a6c-b931-bf7dda9ec50e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:15:50.056991 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:50.056924    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1695694b-12a1-4a6c-b931-bf7dda9ec50e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:15:50.056991 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:50.056985    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6207663-1cfa-48d9-8382-12973bbf29fd-metrics-tls\") pod \"dns-default-fl2tq\" (UID: \"a6207663-1cfa-48d9-8382-12973bbf29fd\") " pod="openshift-dns/dns-default-fl2tq"
Oct 31 21:15:50.057092 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:50.057054    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 21:15:50.057092 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:50.057063    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 21:15:50.057092 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:50.057068    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 31 21:15:50.057180 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:50.057113    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6207663-1cfa-48d9-8382-12973bbf29fd-metrics-tls podName:a6207663-1cfa-48d9-8382-12973bbf29fd nodeName:}" failed. No retries permitted until 2026-10-31 21:15:51.057096493 +0000 UTC m=+31.162097385 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6207663-1cfa-48d9-8382-12973bbf29fd-metrics-tls") pod "dns-default-fl2tq" (UID: "a6207663-1cfa-48d9-8382-12973bbf29fd") : secret "dns-default-metrics-tls" not found
Oct 31 21:15:50.057180 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:50.057126    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1695694b-12a1-4a6c-b931-bf7dda9ec50e-kube-rbac-proxy-cm podName:1695694b-12a1-4a6c-b931-bf7dda9ec50e nodeName:}" failed. No retries permitted until 2026-10-31 21:15:51.057120276 +0000 UTC m=+31.162121166 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1695694b-12a1-4a6c-b931-bf7dda9ec50e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-47zb8" (UID: "1695694b-12a1-4a6c-b931-bf7dda9ec50e") : configmap "kube-rbac-proxy" not found
Oct 31 21:15:50.057180 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:50.057137    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1695694b-12a1-4a6c-b931-bf7dda9ec50e-insights-runtime-extractor-tls podName:1695694b-12a1-4a6c-b931-bf7dda9ec50e nodeName:}" failed. No retries permitted until 2026-10-31 21:15:51.057130223 +0000 UTC m=+31.162131114 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1695694b-12a1-4a6c-b931-bf7dda9ec50e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-47zb8" (UID: "1695694b-12a1-4a6c-b931-bf7dda9ec50e") : secret "insights-runtime-extractor-tls" not found
Oct 31 21:15:50.717622 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:50.717582    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-x8cxm" event={"ID":"6d63e78f-f474-4966-adb5-401be54f4ab4","Type":"ContainerStarted","Data":"2b851e26be51e365888088ee65bfb2f98feba4569e308ae07cc00b5b172091c4"}
Oct 31 21:15:51.065060 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:51.065029    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6207663-1cfa-48d9-8382-12973bbf29fd-metrics-tls\") pod \"dns-default-fl2tq\" (UID: \"a6207663-1cfa-48d9-8382-12973bbf29fd\") " pod="openshift-dns/dns-default-fl2tq"
Oct 31 21:15:51.065255 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:51.065114    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1695694b-12a1-4a6c-b931-bf7dda9ec50e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:15:51.065255 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:51.065165    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1695694b-12a1-4a6c-b931-bf7dda9ec50e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:15:51.065255 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:51.065185    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 21:15:51.065255 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:51.065246    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6207663-1cfa-48d9-8382-12973bbf29fd-metrics-tls podName:a6207663-1cfa-48d9-8382-12973bbf29fd nodeName:}" failed. No retries permitted until 2026-10-31 21:15:53.065229108 +0000 UTC m=+33.170230016 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6207663-1cfa-48d9-8382-12973bbf29fd-metrics-tls") pod "dns-default-fl2tq" (UID: "a6207663-1cfa-48d9-8382-12973bbf29fd") : secret "dns-default-metrics-tls" not found
Oct 31 21:15:51.065255 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:51.065253    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 31 21:15:51.065509 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:51.065278    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 21:15:51.065509 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:51.065287    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1695694b-12a1-4a6c-b931-bf7dda9ec50e-kube-rbac-proxy-cm podName:1695694b-12a1-4a6c-b931-bf7dda9ec50e nodeName:}" failed. No retries permitted until 2026-10-31 21:15:53.065278027 +0000 UTC m=+33.170278935 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1695694b-12a1-4a6c-b931-bf7dda9ec50e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-47zb8" (UID: "1695694b-12a1-4a6c-b931-bf7dda9ec50e") : configmap "kube-rbac-proxy" not found
Oct 31 21:15:51.065509 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:51.065358    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1695694b-12a1-4a6c-b931-bf7dda9ec50e-insights-runtime-extractor-tls podName:1695694b-12a1-4a6c-b931-bf7dda9ec50e nodeName:}" failed. No retries permitted until 2026-10-31 21:15:53.065340879 +0000 UTC m=+33.170341775 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1695694b-12a1-4a6c-b931-bf7dda9ec50e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-47zb8" (UID: "1695694b-12a1-4a6c-b931-bf7dda9ec50e") : secret "insights-runtime-extractor-tls" not found
Oct 31 21:15:51.130202 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:51.130164    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:51.332732 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:51.332654    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:51.477754 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:51.477728    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:51.655603 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:51.655516    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:52.248039 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:52.248007    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wx67m_bb42ddd8-d246-4bf3-ae35-27aa4584b163/dns-node-resolver/0.log"
Oct 31 21:15:52.304480 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:52.304444    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:52.723133 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:52.723105    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-x8cxm" event={"ID":"6d63e78f-f474-4966-adb5-401be54f4ab4","Type":"ContainerStarted","Data":"10f42c00cb740f4aa09d36bd4a6d75581f534cbf1ff9fc208fbdf6efb44d0a1d"}
Oct 31 21:15:52.726836 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:52.726805    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:52.899380 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:52.899303    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:53.079293 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:53.079266    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1695694b-12a1-4a6c-b931-bf7dda9ec50e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:15:53.079428 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:53.079310    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6207663-1cfa-48d9-8382-12973bbf29fd-metrics-tls\") pod \"dns-default-fl2tq\" (UID: \"a6207663-1cfa-48d9-8382-12973bbf29fd\") " pod="openshift-dns/dns-default-fl2tq"
Oct 31 21:15:53.079428 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:53.079346    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1695694b-12a1-4a6c-b931-bf7dda9ec50e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:15:53.079428 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:53.079374    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 31 21:15:53.079428 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:53.079426    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1695694b-12a1-4a6c-b931-bf7dda9ec50e-kube-rbac-proxy-cm podName:1695694b-12a1-4a6c-b931-bf7dda9ec50e nodeName:}" failed. No retries permitted until 2026-10-31 21:15:57.079412922 +0000 UTC m=+37.184413814 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1695694b-12a1-4a6c-b931-bf7dda9ec50e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-47zb8" (UID: "1695694b-12a1-4a6c-b931-bf7dda9ec50e") : configmap "kube-rbac-proxy" not found
Oct 31 21:15:53.079562 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:53.079473    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 21:15:53.079562 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:53.079505    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 21:15:53.079562 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:53.079534    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6207663-1cfa-48d9-8382-12973bbf29fd-metrics-tls podName:a6207663-1cfa-48d9-8382-12973bbf29fd nodeName:}" failed. No retries permitted until 2026-10-31 21:15:57.079516737 +0000 UTC m=+37.184517647 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a6207663-1cfa-48d9-8382-12973bbf29fd-metrics-tls") pod "dns-default-fl2tq" (UID: "a6207663-1cfa-48d9-8382-12973bbf29fd") : secret "dns-default-metrics-tls" not found
Oct 31 21:15:53.079703 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:53.079551    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1695694b-12a1-4a6c-b931-bf7dda9ec50e-insights-runtime-extractor-tls podName:1695694b-12a1-4a6c-b931-bf7dda9ec50e nodeName:}" failed. No retries permitted until 2026-10-31 21:15:57.079541403 +0000 UTC m=+37.184542297 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1695694b-12a1-4a6c-b931-bf7dda9ec50e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-47zb8" (UID: "1695694b-12a1-4a6c-b931-bf7dda9ec50e") : secret "insights-runtime-extractor-tls" not found
Oct 31 21:15:53.104735 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:53.104715    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:53.214748 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:53.214724    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:53.247823 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:53.247801    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xhzwc_23da9927-8255-4380-a3f0-b7ea9298ad40/node-ca/0.log"
Oct 31 21:15:53.329483 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:53.329463    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:53.699251 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:53.699222    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:53.770008 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:53.769966    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-x8cxm" podStartSLOduration=2.055680947 podStartE2EDuration="4.769934781s" totalImagesPullingTime="2.714253834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:15:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:15:53.769743609 +0000 UTC m=+33.874744521" watchObservedRunningTime="2026-10-31 21:15:53.769934781 +0000 UTC m=+33.874935693"
Oct 31 21:15:53.825162 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:53.825132    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:53.848867 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:53.848845    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-x8cxm_6d63e78f-f474-4966-adb5-401be54f4ab4/serve-healthcheck-canary/0.log"
Oct 31 21:15:54.184474 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:54.184400    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-original-pull-secret\") pod \"global-pull-secret-syncer-9zw4h\" (UID: \"22daf2b3-91ca-462b-8714-0e0b0ebbcb88\") " pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:15:54.184619 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:54.184549    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e254918-3cc1-4419-9820-37dff30f4445-metrics-certs\") pod \"network-metrics-daemon-bwhxr\" (UID: \"3e254918-3cc1-4419-9820-37dff30f4445\") " pod="openshift-multus/network-metrics-daemon-bwhxr"
Oct 31 21:15:54.184663 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:54.184635    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 31 21:15:54.184702 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:54.184684    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e254918-3cc1-4419-9820-37dff30f4445-metrics-certs podName:3e254918-3cc1-4419-9820-37dff30f4445 nodeName:}" failed. No retries permitted until 2026-10-31 21:16:26.184672524 +0000 UTC m=+66.289673414 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3e254918-3cc1-4419-9820-37dff30f4445-metrics-certs") pod "network-metrics-daemon-bwhxr" (UID: "3e254918-3cc1-4419-9820-37dff30f4445") : secret "metrics-daemon-secret" not found
Oct 31 21:15:54.186583 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:54.186557    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-original-pull-secret\") pod \"global-pull-secret-syncer-9zw4h\" (UID: \"22daf2b3-91ca-462b-8714-0e0b0ebbcb88\") " pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:15:54.269224 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:54.269199    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:15:54.285641 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:54.285613    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfq9g\" (UniqueName: \"kubernetes.io/projected/08d86dfb-548b-4a83-a1c2-d6ce8d4a0091-kube-api-access-pfq9g\") pod \"network-check-target-bsvln\" (UID: \"08d86dfb-548b-4a83-a1c2-d6ce8d4a0091\") " pod="openshift-network-diagnostics/network-check-target-bsvln"
Oct 31 21:15:54.287981 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:54.287965    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfq9g\" (UniqueName: \"kubernetes.io/projected/08d86dfb-548b-4a83-a1c2-d6ce8d4a0091-kube-api-access-pfq9g\") pod \"network-check-target-bsvln\" (UID: \"08d86dfb-548b-4a83-a1c2-d6ce8d4a0091\") " pod="openshift-network-diagnostics/network-check-target-bsvln"
Oct 31 21:15:54.533747 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:54.533710    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:54.563645 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:54.563624    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bsvln"
Oct 31 21:15:54.634907 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:54.634879    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9zw4h"]
Oct 31 21:15:54.645172 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:54.645116    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:54.729136 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:54.729105    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9zw4h" event={"ID":"22daf2b3-91ca-462b-8714-0e0b0ebbcb88","Type":"ContainerStarted","Data":"487ce8cd1e5f8b6ff2ebde6dd1c41b0ce1adf5472b52c672d688796311e56c99"}
Oct 31 21:15:54.757115 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:54.757085    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:54.882395 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:54.882331    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bsvln"]
Oct 31 21:15:54.887712 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:54.887674    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08d86dfb_548b_4a83_a1c2_d6ce8d4a0091.slice/crio-9c2e2fda75e9ea6d6199dd288dc22f34b354dfe11647c8a42d67e72ac14c90fa WatchSource:0}: Error finding container 9c2e2fda75e9ea6d6199dd288dc22f34b354dfe11647c8a42d67e72ac14c90fa: Status 404 returned error can't find the container with id 9c2e2fda75e9ea6d6199dd288dc22f34b354dfe11647c8a42d67e72ac14c90fa
Oct 31 21:15:54.899106 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:54.899084    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:55.098850 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:55.098818    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:55.299182 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:55.299138    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:55.733655 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:55.733580    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bsvln" event={"ID":"08d86dfb-548b-4a83-a1c2-d6ce8d4a0091","Type":"ContainerStarted","Data":"9c2e2fda75e9ea6d6199dd288dc22f34b354dfe11647c8a42d67e72ac14c90fa"}
Oct 31 21:15:56.502295 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:56.502259    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:56.705748 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:56.705684    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:56.900469 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:56.900436    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:57.100205 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:57.100156    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:57.110964 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:57.110197    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6207663-1cfa-48d9-8382-12973bbf29fd-metrics-tls\") pod \"dns-default-fl2tq\" (UID: \"a6207663-1cfa-48d9-8382-12973bbf29fd\") " pod="openshift-dns/dns-default-fl2tq"
Oct 31 21:15:57.110964 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:57.110246    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1695694b-12a1-4a6c-b931-bf7dda9ec50e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:15:57.110964 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:57.110288    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1695694b-12a1-4a6c-b931-bf7dda9ec50e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:15:57.110964 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:57.110399    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 21:15:57.110964 ip-10-0-136-157 kubenswrapper[2575]: E1031 21:15:57.110512    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1695694b-12a1-4a6c-b931-bf7dda9ec50e-insights-runtime-extractor-tls podName:1695694b-12a1-4a6c-b931-bf7dda9ec50e nodeName:}" failed. No retries permitted until 2026-10-31 21:16:05.110471753 +0000 UTC m=+45.215472649 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1695694b-12a1-4a6c-b931-bf7dda9ec50e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-47zb8" (UID: "1695694b-12a1-4a6c-b931-bf7dda9ec50e") : secret "insights-runtime-extractor-tls" not found
Oct 31 21:15:57.110964 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:57.110895    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1695694b-12a1-4a6c-b931-bf7dda9ec50e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:15:57.113049 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:57.113019    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6207663-1cfa-48d9-8382-12973bbf29fd-metrics-tls\") pod \"dns-default-fl2tq\" (UID: \"a6207663-1cfa-48d9-8382-12973bbf29fd\") " pod="openshift-dns/dns-default-fl2tq"
Oct 31 21:15:57.139276 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:57.139251    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fl2tq"
Oct 31 21:15:57.301861 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:57.301826    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:57.477383 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:57.477304    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fl2tq"]
Oct 31 21:15:57.482475 ip-10-0-136-157 kubenswrapper[2575]: W1031 21:15:57.482439    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6207663_1cfa_48d9_8382_12973bbf29fd.slice/crio-f39f443d11426c0cd424f5ab150a5868364bc141077e7e513446918f3376f490 WatchSource:0}: Error finding container f39f443d11426c0cd424f5ab150a5868364bc141077e7e513446918f3376f490: Status 404 returned error can't find the container with id f39f443d11426c0cd424f5ab150a5868364bc141077e7e513446918f3376f490
Oct 31 21:15:57.740815 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:57.740733    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fl2tq" event={"ID":"a6207663-1cfa-48d9-8382-12973bbf29fd","Type":"ContainerStarted","Data":"f39f443d11426c0cd424f5ab150a5868364bc141077e7e513446918f3376f490"}
Oct 31 21:15:58.500925 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:58.500896    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:58.698842 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:58.698721    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:58.745343 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:58.745273    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bsvln" event={"ID":"08d86dfb-548b-4a83-a1c2-d6ce8d4a0091","Type":"ContainerStarted","Data":"c116fec0b9cc2fe1614e5f7474dfdf4c4dc7fd0ef1986b3dfa3b2e33f60092c6"}
Oct 31 21:15:58.745715 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:58.745504    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-bsvln"
Oct 31 21:15:58.785904 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:58.785851    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-bsvln" podStartSLOduration=35.28561981 podStartE2EDuration="38.785837613s" totalImagesPullingTime="3.500217803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:15:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:15:58.784949929 +0000 UTC m=+38.889950834" watchObservedRunningTime="2026-10-31 21:15:58.785837613 +0000 UTC m=+38.890838523"
Oct 31 21:15:58.902872 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:58.902833    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:59.100372 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:59.100332    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:59.300378 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:59.300349    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:15:59.534432 ip-10-0-136-157 kubenswrapper[2575]: I1031 21:15:59.534404    2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:00.026568 ip-10-0-136-157 systemd[1]: Stopping Kubernetes Kubelet...
Oct 31 21:16:00.039292 ip-10-0-136-157 systemd[1]: kubelet.service: Deactivated successfully.
Oct 31 21:16:00.039461 ip-10-0-136-157 systemd[1]: Stopped Kubernetes Kubelet.
Oct 31 21:16:00.039511 ip-10-0-136-157 systemd[1]: kubelet.service: Consumed 2.263s CPU time.
Oct 31 21:16:00.055543 ip-10-0-136-157 systemd[1]: Starting Kubernetes Kubelet...
Oct 31 21:16:00.140675 ip-10-0-136-157 kubenswrapper[4460]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 21:16:00.140675 ip-10-0-136-157 kubenswrapper[4460]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 31 21:16:00.140675 ip-10-0-136-157 kubenswrapper[4460]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 21:16:00.140675 ip-10-0-136-157 kubenswrapper[4460]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 21:16:00.142746 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142674    4460 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 21:16:00.142746 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142746    4460 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 21:16:00.142813 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142751    4460 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 21:16:00.142813 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142755    4460 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 21:16:00.142813 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142758    4460 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 21:16:00.142813 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142760    4460 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 21:16:00.142813 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142764    4460 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 21:16:00.142813 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142766    4460 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 21:16:00.142813 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142769    4460 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 21:16:00.142813 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142772    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 21:16:00.142813 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142774    4460 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 21:16:00.142813 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142777    4460 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 21:16:00.142813 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142779    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 21:16:00.142813 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142782    4460 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 21:16:00.142813 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142785    4460 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 21:16:00.142813 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142788    4460 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 21:16:00.142813 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142790    4460 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 21:16:00.142813 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142793    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 21:16:00.142813 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142795    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 21:16:00.142813 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142798    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 21:16:00.142813 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142801    4460 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 21:16:00.142813 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142809    4460 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 21:16:00.143342 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142812    4460 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 21:16:00.143342 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142815    4460 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 21:16:00.143342 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142818    4460 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 21:16:00.143342 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142821    4460 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 21:16:00.143342 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142824    4460 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 21:16:00.143342 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142827    4460 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 21:16:00.143342 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142830    4460 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 21:16:00.143342 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142832    4460 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 21:16:00.143342 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142835    4460 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 21:16:00.143342 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142838    4460 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 21:16:00.143342 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142840    4460 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 21:16:00.143342 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142843    4460 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 21:16:00.143342 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142845    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 21:16:00.143342 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142848    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 21:16:00.143342 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142850    4460 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 21:16:00.143342 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142853    4460 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 21:16:00.143342 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142856    4460 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 21:16:00.143342 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142858    4460 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 21:16:00.143342 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142861    4460 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 21:16:00.143805 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142863    4460 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 21:16:00.143805 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142866    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 21:16:00.143805 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142869    4460 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 21:16:00.143805 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142871    4460 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 21:16:00.143805 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142874    4460 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 21:16:00.143805 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142877    4460 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 21:16:00.143805 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142879    4460 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 21:16:00.143805 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142882    4460 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 21:16:00.143805 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142884    4460 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 21:16:00.143805 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142887    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 21:16:00.143805 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142890    4460 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 21:16:00.143805 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142893    4460 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 21:16:00.143805 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142895    4460 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 21:16:00.143805 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142898    4460 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 21:16:00.143805 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142901    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 21:16:00.143805 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142904    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 21:16:00.143805 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142906    4460 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 21:16:00.143805 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142909    4460 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 21:16:00.143805 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142911    4460 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 21:16:00.143805 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142914    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 21:16:00.144418 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142917    4460 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 21:16:00.144418 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142919    4460 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 21:16:00.144418 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142922    4460 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 21:16:00.144418 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142925    4460 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 21:16:00.144418 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142927    4460 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 21:16:00.144418 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142929    4460 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 21:16:00.144418 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142934    4460 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 21:16:00.144418 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142951    4460 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 21:16:00.144418 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142955    4460 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 21:16:00.144418 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142957    4460 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 21:16:00.144418 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142960    4460 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 21:16:00.144418 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142963    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 21:16:00.144418 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142966    4460 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 21:16:00.144418 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142969    4460 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 21:16:00.144418 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142972    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 21:16:00.144418 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142975    4460 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 21:16:00.144418 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142979    4460 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 21:16:00.144418 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142982    4460 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 21:16:00.144418 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142985    4460 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 21:16:00.144418 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142987    4460 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 21:16:00.144907 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142990    4460 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 21:16:00.144907 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142992    4460 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 21:16:00.144907 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142995    4460 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 21:16:00.144907 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.142998    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 21:16:00.144907 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143008    4460 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 21:16:00.144907 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143011    4460 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 21:16:00.144907 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143015    4460 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 21:16:00.144907 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143018    4460 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 21:16:00.144907 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143021    4460 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 21:16:00.144907 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143024    4460 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 21:16:00.144907 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143026    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 21:16:00.144907 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143035    4460 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 21:16:00.144907 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143038    4460 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 21:16:00.144907 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143041    4460 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 21:16:00.144907 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143043    4460 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 21:16:00.144907 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143046    4460 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 21:16:00.144907 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143048    4460 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 21:16:00.144907 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143051    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 21:16:00.144907 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143054    4460 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 21:16:00.145408 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143056    4460 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 21:16:00.145408 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143059    4460 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 21:16:00.145408 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143062    4460 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 21:16:00.145408 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143064    4460 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 21:16:00.145408 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143067    4460 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 21:16:00.145408 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143069    4460 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 21:16:00.145408 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143829    4460 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 21:16:00.145408 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143836    4460 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 21:16:00.145408 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143840    4460 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 21:16:00.145408 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143842    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 21:16:00.145408 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143845    4460 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 21:16:00.145408 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143848    4460 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 21:16:00.145408 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143851    4460 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 21:16:00.145408 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143854    4460 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 21:16:00.145408 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143856    4460 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 21:16:00.145408 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143859    4460 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 21:16:00.145408 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143862    4460 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 21:16:00.145408 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143864    4460 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 21:16:00.145408 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143868    4460 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 21:16:00.145408 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143872    4460 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 21:16:00.145894 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143876    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 21:16:00.145894 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143881    4460 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 21:16:00.145894 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143884    4460 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 21:16:00.145894 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143887    4460 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 21:16:00.145894 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143890    4460 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 21:16:00.145894 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143893    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 21:16:00.145894 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143895    4460 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 21:16:00.145894 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143898    4460 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 21:16:00.145894 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143901    4460 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 21:16:00.145894 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143903    4460 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 21:16:00.145894 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143906    4460 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 21:16:00.145894 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143909    4460 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 21:16:00.145894 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143912    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 21:16:00.145894 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143914    4460 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 21:16:00.145894 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143917    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 21:16:00.145894 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143920    4460 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 21:16:00.145894 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143923    4460 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 21:16:00.145894 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143926    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 21:16:00.145894 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143929    4460 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 21:16:00.146374 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143932    4460 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 21:16:00.146374 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143934    4460 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 21:16:00.146374 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143951    4460 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 21:16:00.146374 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143954    4460 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 21:16:00.146374 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143957    4460 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 21:16:00.146374 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143959    4460 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 21:16:00.146374 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143962    4460 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 21:16:00.146374 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143965    4460 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 21:16:00.146374 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143968    4460 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 21:16:00.146374 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143971    4460 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 21:16:00.146374 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143973    4460 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 21:16:00.146374 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143976    4460 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 21:16:00.146374 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143982    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 21:16:00.146374 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143985    4460 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 21:16:00.146374 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143988    4460 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 21:16:00.146374 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143990    4460 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 21:16:00.146374 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143993    4460 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 21:16:00.146374 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.143997    4460 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 21:16:00.146374 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144000    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 21:16:00.146374 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144002    4460 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 21:16:00.146869 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144005    4460 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 21:16:00.146869 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144008    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 21:16:00.146869 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144011    4460 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 21:16:00.146869 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144013    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 21:16:00.146869 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144016    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 21:16:00.146869 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144019    4460 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 21:16:00.146869 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144021    4460 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 21:16:00.146869 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144024    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 21:16:00.146869 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144027    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 21:16:00.146869 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144030    4460 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 21:16:00.146869 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144032    4460 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 21:16:00.146869 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144035    4460 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 21:16:00.146869 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144038    4460 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 21:16:00.146869 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144040    4460 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 21:16:00.146869 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144043    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 21:16:00.146869 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144046    4460 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 21:16:00.146869 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144049    4460 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 21:16:00.146869 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144051    4460 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 21:16:00.146869 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144054    4460 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 21:16:00.147389 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144056    4460 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 21:16:00.147389 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144059    4460 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 21:16:00.147389 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144062    4460 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 21:16:00.147389 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144064    4460 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 21:16:00.147389 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144067    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 21:16:00.147389 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144069    4460 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 21:16:00.147389 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144077    4460 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 21:16:00.147389 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144080    4460 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 21:16:00.147389 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144083    4460 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 21:16:00.147389 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144085    4460 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 21:16:00.147389 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144088    4460 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 21:16:00.147389 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144091    4460 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 21:16:00.147389 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144093    4460 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 21:16:00.147389 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144096    4460 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 21:16:00.147389 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144098    4460 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 21:16:00.147389 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144101    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 21:16:00.147389 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144104    4460 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 21:16:00.147389 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144106    4460 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 21:16:00.147389 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144109    4460 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 21:16:00.147958 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144112    4460 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 21:16:00.147958 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144114    4460 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 21:16:00.147958 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144117    4460 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 21:16:00.147958 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144122    4460 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 21:16:00.147958 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144125    4460 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 21:16:00.147958 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144127    4460 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 21:16:00.147958 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144130    4460 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 21:16:00.147958 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144133    4460 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 21:16:00.147958 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144136    4460 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 21:16:00.147958 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144138    4460 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 21:16:00.147958 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144141    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 21:16:00.147958 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144144    4460 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 21:16:00.147958 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144146    4460 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 21:16:00.147958 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144149    4460 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 21:16:00.147958 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144151    4460 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 21:16:00.147958 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144657    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 21:16:00.147958 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144662    4460 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 21:16:00.147958 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144665    4460 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 21:16:00.147958 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144668    4460 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 21:16:00.147958 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144671    4460 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 21:16:00.148515 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144674    4460 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 21:16:00.148515 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144677    4460 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 21:16:00.148515 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144679    4460 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 21:16:00.148515 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144682    4460 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 21:16:00.148515 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144684    4460 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 21:16:00.148515 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144687    4460 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 21:16:00.148515 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144690    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 21:16:00.148515 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144693    4460 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 21:16:00.148515 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144695    4460 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 21:16:00.148515 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144698    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 21:16:00.148515 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144701    4460 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 21:16:00.148515 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144703    4460 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 21:16:00.148515 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144706    4460 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 21:16:00.148515 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144708    4460 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 21:16:00.148515 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144711    4460 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 21:16:00.148515 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144718    4460 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 21:16:00.148515 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144721    4460 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 21:16:00.148515 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144723    4460 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 21:16:00.148515 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144726    4460 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 21:16:00.148515 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144729    4460 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 21:16:00.149167 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144731    4460 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 21:16:00.149167 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144734    4460 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 21:16:00.149167 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144737    4460 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 21:16:00.149167 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144739    4460 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 21:16:00.149167 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144742    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 21:16:00.149167 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144745    4460 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 21:16:00.149167 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144747    4460 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 21:16:00.149167 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144750    4460 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 21:16:00.149167 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144752    4460 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 21:16:00.149167 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144755    4460 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 21:16:00.149167 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144758    4460 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 21:16:00.149167 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144760    4460 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 21:16:00.149167 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144763    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 21:16:00.149167 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144766    4460 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 21:16:00.149167 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144769    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 21:16:00.149167 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144773    4460 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 21:16:00.149167 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144779    4460 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 21:16:00.149167 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144782    4460 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 21:16:00.149167 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144785    4460 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 21:16:00.149899 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144788    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 21:16:00.149899 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144790    4460 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 21:16:00.149899 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144793    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 21:16:00.149899 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144796    4460 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 21:16:00.149899 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144798    4460 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 21:16:00.149899 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144800    4460 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 21:16:00.149899 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144803    4460 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 21:16:00.149899 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144805    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 21:16:00.149899 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144810    4460 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 21:16:00.149899 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144813    4460 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 21:16:00.149899 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144816    4460 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 21:16:00.149899 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144819    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 21:16:00.149899 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144821    4460 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 21:16:00.149899 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144824    4460 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 21:16:00.149899 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144826    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 21:16:00.149899 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144829    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 21:16:00.149899 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144832    4460 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 21:16:00.149899 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144834    4460 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 21:16:00.149899 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144837    4460 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 21:16:00.149899 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144839    4460 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 21:16:00.150412 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144842    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 21:16:00.150412 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144844    4460 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 21:16:00.150412 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144847    4460 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 21:16:00.150412 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144849    4460 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 21:16:00.150412 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144853    4460 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 21:16:00.150412 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144856    4460 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 21:16:00.150412 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144859    4460 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 21:16:00.150412 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144862    4460 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 21:16:00.150412 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144864    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 21:16:00.150412 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144868    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 21:16:00.150412 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144871    4460 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 21:16:00.150412 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144873    4460 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 21:16:00.150412 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144876    4460 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 21:16:00.150412 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144878    4460 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 21:16:00.150412 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144881    4460 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 21:16:00.150412 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144884    4460 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 21:16:00.150412 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144886    4460 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 21:16:00.150412 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144889    4460 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 21:16:00.150412 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144891    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 21:16:00.150882 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144894    4460 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 21:16:00.150882 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144898    4460 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 21:16:00.150882 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144900    4460 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 21:16:00.150882 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144903    4460 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 21:16:00.150882 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144905    4460 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 21:16:00.150882 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144908    4460 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 21:16:00.150882 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144911    4460 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 21:16:00.150882 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144914    4460 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 21:16:00.150882 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144916    4460 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 21:16:00.150882 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144919    4460 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 21:16:00.150882 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144921    4460 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 21:16:00.150882 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144924    4460 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 21:16:00.150882 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144927    4460 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 21:16:00.150882 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144929    4460 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 21:16:00.150882 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144932    4460 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 21:16:00.150882 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144934    4460 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 21:16:00.150882 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144950    4460 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 21:16:00.150882 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144953    4460 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 21:16:00.150882 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144956    4460 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 21:16:00.151416 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144960    4460 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 21:16:00.151416 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144964    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 21:16:00.151416 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144968    4460 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 21:16:00.151416 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.144972    4460 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 21:16:00.151416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145063    4460 flags.go:64] FLAG: --address="0.0.0.0"
Oct 31 21:16:00.151416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145071    4460 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 31 21:16:00.151416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145078    4460 flags.go:64] FLAG: --anonymous-auth="true"
Oct 31 21:16:00.151416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145082    4460 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 31 21:16:00.151416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145087    4460 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 31 21:16:00.151416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145091    4460 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 31 21:16:00.151416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145095    4460 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 31 21:16:00.151416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145100    4460 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 31 21:16:00.151416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145103    4460 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 31 21:16:00.151416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145106    4460 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 31 21:16:00.151416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145111    4460 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 31 21:16:00.151416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145115    4460 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 31 21:16:00.151416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145118    4460 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 31 21:16:00.151416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145121    4460 flags.go:64] FLAG: --cgroup-root=""
Oct 31 21:16:00.151416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145124    4460 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 31 21:16:00.151416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145127    4460 flags.go:64] FLAG: --client-ca-file=""
Oct 31 21:16:00.151416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145130    4460 flags.go:64] FLAG: --cloud-provider="external"
Oct 31 21:16:00.151416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145133    4460 flags.go:64] FLAG: --cluster-dns="[]"
Oct 31 21:16:00.151416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145136    4460 flags.go:64] FLAG: --cluster-domain=""
Oct 31 21:16:00.151990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145140    4460 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 31 21:16:00.151990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145143    4460 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 31 21:16:00.151990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145146    4460 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 31 21:16:00.151990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145150    4460 flags.go:64] FLAG: --container-log-max-files="5"
Oct 31 21:16:00.151990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145154    4460 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 31 21:16:00.151990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145157    4460 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 31 21:16:00.151990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145160    4460 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 31 21:16:00.151990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145164    4460 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 31 21:16:00.151990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145166    4460 flags.go:64] FLAG: --contention-profiling="false"
Oct 31 21:16:00.151990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145169    4460 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 31 21:16:00.151990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145172    4460 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 31 21:16:00.151990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145176    4460 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 31 21:16:00.151990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145180    4460 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 31 21:16:00.151990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145184    4460 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 31 21:16:00.151990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145187    4460 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 31 21:16:00.151990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145190    4460 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 31 21:16:00.151990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145193    4460 flags.go:64] FLAG: --enable-load-reader="false"
Oct 31 21:16:00.151990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145196    4460 flags.go:64] FLAG: --enable-server="true"
Oct 31 21:16:00.151990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145199    4460 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 31 21:16:00.151990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145204    4460 flags.go:64] FLAG: --event-burst="100"
Oct 31 21:16:00.151990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145207    4460 flags.go:64] FLAG: --event-qps="50"
Oct 31 21:16:00.151990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145210    4460 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 31 21:16:00.151990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145213    4460 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 31 21:16:00.151990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145217    4460 flags.go:64] FLAG: --eviction-hard=""
Oct 31 21:16:00.152589 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145222    4460 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 31 21:16:00.152589 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145225    4460 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 31 21:16:00.152589 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145228    4460 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 31 21:16:00.152589 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145231    4460 flags.go:64] FLAG: --eviction-soft=""
Oct 31 21:16:00.152589 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145235    4460 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 31 21:16:00.152589 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145238    4460 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 31 21:16:00.152589 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145241    4460 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 31 21:16:00.152589 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145244    4460 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 31 21:16:00.152589 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145247    4460 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 31 21:16:00.152589 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145250    4460 flags.go:64] FLAG: --fail-swap-on="true"
Oct 31 21:16:00.152589 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145253    4460 flags.go:64] FLAG: --feature-gates=""
Oct 31 21:16:00.152589 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145258    4460 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 31 21:16:00.152589 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145261    4460 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 31 21:16:00.152589 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145264    4460 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 31 21:16:00.152589 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145267    4460 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 31 21:16:00.152589 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145270    4460 flags.go:64] FLAG: --healthz-port="10248"
Oct 31 21:16:00.152589 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145273    4460 flags.go:64] FLAG: --help="false"
Oct 31 21:16:00.152589 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145276    4460 flags.go:64] FLAG: --hostname-override="ip-10-0-136-157.ec2.internal"
Oct 31 21:16:00.152589 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145279    4460 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 31 21:16:00.152589 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145282    4460 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 31 21:16:00.152589 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145286    4460 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 31 21:16:00.152589 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145290    4460 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 31 21:16:00.152589 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145294    4460 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 31 21:16:00.153153 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145297    4460 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 31 21:16:00.153153 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145299    4460 flags.go:64] FLAG: --image-service-endpoint=""
Oct 31 21:16:00.153153 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145302    4460 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 31 21:16:00.153153 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145306    4460 flags.go:64] FLAG: --kube-api-burst="100"
Oct 31 21:16:00.153153 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145309    4460 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 31 21:16:00.153153 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145312    4460 flags.go:64] FLAG: --kube-api-qps="50"
Oct 31 21:16:00.153153 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145316    4460 flags.go:64] FLAG: --kube-reserved=""
Oct 31 21:16:00.153153 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145319    4460 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 31 21:16:00.153153 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145323    4460 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 31 21:16:00.153153 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145327    4460 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 31 21:16:00.153153 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145330    4460 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 31 21:16:00.153153 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145333    4460 flags.go:64] FLAG: --lock-file=""
Oct 31 21:16:00.153153 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145336    4460 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 31 21:16:00.153153 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145339    4460 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 31 21:16:00.153153 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145342    4460 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 31 21:16:00.153153 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145348    4460 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 31 21:16:00.153153 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145351    4460 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 31 21:16:00.153153 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145354    4460 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 31 21:16:00.153153 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145357    4460 flags.go:64] FLAG: --logging-format="text"
Oct 31 21:16:00.153153 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145360    4460 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 31 21:16:00.153153 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145364    4460 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 31 21:16:00.153153 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145367    4460 flags.go:64] FLAG: --manifest-url=""
Oct 31 21:16:00.153153 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145370    4460 flags.go:64] FLAG: --manifest-url-header=""
Oct 31 21:16:00.153153 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145374    4460 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 31 21:16:00.153153 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145377    4460 flags.go:64] FLAG: --max-open-files="1000000"
Oct 31 21:16:00.153745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145400    4460 flags.go:64] FLAG: --max-pods="110"
Oct 31 21:16:00.153745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145404    4460 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 31 21:16:00.153745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145409    4460 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 31 21:16:00.153745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145413    4460 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 31 21:16:00.153745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145416    4460 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 31 21:16:00.153745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145420    4460 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 31 21:16:00.153745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145423    4460 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 31 21:16:00.153745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145426    4460 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 31 21:16:00.153745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145434    4460 flags.go:64] FLAG: --node-status-max-images="50"
Oct 31 21:16:00.153745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145437    4460 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 31 21:16:00.153745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145440    4460 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 31 21:16:00.153745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145443    4460 flags.go:64] FLAG: --pod-cidr=""
Oct 31 21:16:00.153745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145446    4460 flags.go:64] FLAG: --pod-manifest-path=""
Oct 31 21:16:00.153745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145450    4460 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 31 21:16:00.153745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145453    4460 flags.go:64] FLAG: --pods-per-core="0"
Oct 31 21:16:00.153745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145456    4460 flags.go:64] FLAG: --port="10250"
Oct 31 21:16:00.153745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145459    4460 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 31 21:16:00.153745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145462    4460 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-067941a7bcb0aa0b3"
Oct 31 21:16:00.153745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145466    4460 flags.go:64] FLAG: --qos-reserved=""
Oct 31 21:16:00.153745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145469    4460 flags.go:64] FLAG: --read-only-port="10255"
Oct 31 21:16:00.153745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145473    4460 flags.go:64] FLAG: --register-node="true"
Oct 31 21:16:00.153745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145475    4460 flags.go:64] FLAG: --register-with-taints=""
Oct 31 21:16:00.153745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145479    4460 flags.go:64] FLAG: --registry-burst="10"
Oct 31 21:16:00.153745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145482    4460 flags.go:64] FLAG: --registry-qps="5"
Oct 31 21:16:00.153745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145485    4460 flags.go:64] FLAG: --reserved-cpus=""
Oct 31 21:16:00.153745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145488    4460 flags.go:64] FLAG: --reserved-memory=""
Oct 31 21:16:00.154400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145491    4460 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 31 21:16:00.154400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145495    4460 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 31 21:16:00.154400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145498    4460 flags.go:64] FLAG: --rotate-certificates="false"
Oct 31 21:16:00.154400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145501    4460 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 31 21:16:00.154400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145504    4460 flags.go:64] FLAG: --runonce="false"
Oct 31 21:16:00.154400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145506    4460 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 31 21:16:00.154400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145510    4460 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 31 21:16:00.154400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145523    4460 flags.go:64] FLAG: --seccomp-default="false"
Oct 31 21:16:00.154400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145528    4460 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 31 21:16:00.154400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145531    4460 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 31 21:16:00.154400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145535    4460 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 31 21:16:00.154400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145538    4460 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 31 21:16:00.154400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145542    4460 flags.go:64] FLAG: --storage-driver-password="root"
Oct 31 21:16:00.154400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145545    4460 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 31 21:16:00.154400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145548    4460 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 31 21:16:00.154400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145551    4460 flags.go:64] FLAG: --storage-driver-user="root"
Oct 31 21:16:00.154400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145554    4460 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 31 21:16:00.154400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145557    4460 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 31 21:16:00.154400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145560    4460 flags.go:64] FLAG: --system-cgroups=""
Oct 31 21:16:00.154400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145563    4460 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 31 21:16:00.154400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145568    4460 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 31 21:16:00.154400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145571    4460 flags.go:64] FLAG: --tls-cert-file=""
Oct 31 21:16:00.154400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145574    4460 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 31 21:16:00.154400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145578    4460 flags.go:64] FLAG: --tls-min-version=""
Oct 31 21:16:00.154400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145581    4460 flags.go:64] FLAG: --tls-private-key-file=""
Oct 31 21:16:00.155027 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145584    4460 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 31 21:16:00.155027 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145587    4460 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 31 21:16:00.155027 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145590    4460 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 31 21:16:00.155027 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145593    4460 flags.go:64] FLAG: --v="2"
Oct 31 21:16:00.155027 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145597    4460 flags.go:64] FLAG: --version="false"
Oct 31 21:16:00.155027 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145602    4460 flags.go:64] FLAG: --vmodule=""
Oct 31 21:16:00.155027 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145606    4460 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 31 21:16:00.155027 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.145609    4460 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 31 21:16:00.155027 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145736    4460 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 21:16:00.155027 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145739    4460 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 21:16:00.155027 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145742    4460 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 21:16:00.155027 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145745    4460 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 21:16:00.155027 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145748    4460 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 21:16:00.155027 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145751    4460 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 21:16:00.155027 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145753    4460 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 21:16:00.155027 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145757    4460 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 21:16:00.155027 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145760    4460 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 21:16:00.155027 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145762    4460 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 21:16:00.155027 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145765    4460 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 21:16:00.155027 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145768    4460 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 21:16:00.155027 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145771    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 21:16:00.155027 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145774    4460 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 21:16:00.155543 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145777    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 21:16:00.155543 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145779    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 21:16:00.155543 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145782    4460 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 21:16:00.155543 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145784    4460 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 21:16:00.155543 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145787    4460 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 21:16:00.155543 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145789    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 21:16:00.155543 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145792    4460 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 21:16:00.155543 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145795    4460 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 21:16:00.155543 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145797    4460 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 21:16:00.155543 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145800    4460 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 21:16:00.155543 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145802    4460 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 21:16:00.155543 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145805    4460 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 21:16:00.155543 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145808    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 21:16:00.155543 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145810    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 21:16:00.155543 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145813    4460 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 21:16:00.155543 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145815    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 21:16:00.155543 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145818    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 21:16:00.155543 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145820    4460 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 21:16:00.155543 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145823    4460 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 21:16:00.156016 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145825    4460 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 21:16:00.156016 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145827    4460 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 21:16:00.156016 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145830    4460 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 21:16:00.156016 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145832    4460 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 21:16:00.156016 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145835    4460 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 21:16:00.156016 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145837    4460 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 21:16:00.156016 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145841    4460 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 21:16:00.156016 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145844    4460 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 21:16:00.156016 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145846    4460 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 21:16:00.156016 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145849    4460 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 21:16:00.156016 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145851    4460 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 21:16:00.156016 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145854    4460 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 21:16:00.156016 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145857    4460 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 21:16:00.156016 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145859    4460 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 21:16:00.156016 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145862    4460 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 21:16:00.156016 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145864    4460 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 21:16:00.156016 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145867    4460 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 21:16:00.156016 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145869    4460 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 21:16:00.156016 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145872    4460 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 21:16:00.156523 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145875    4460 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 21:16:00.156523 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145877    4460 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 21:16:00.156523 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145880    4460 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 21:16:00.156523 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145882    4460 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 21:16:00.156523 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145885    4460 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 21:16:00.156523 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145889    4460 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 21:16:00.156523 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145892    4460 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 21:16:00.156523 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145895    4460 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 21:16:00.156523 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145898    4460 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 21:16:00.156523 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145901    4460 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 21:16:00.156523 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145904    4460 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 21:16:00.156523 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145907    4460 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 21:16:00.156523 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145910    4460 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 21:16:00.156523 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145912    4460 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 21:16:00.156523 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145915    4460 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 21:16:00.156523 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145917    4460 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 21:16:00.156523 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145920    4460 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 21:16:00.156523 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145922    4460 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 21:16:00.156523 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145925    4460 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 21:16:00.157072 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145930    4460 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 21:16:00.157072 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145932    4460 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 21:16:00.157072 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145948    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 21:16:00.157072 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145951    4460 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 21:16:00.157072 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145954    4460 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 21:16:00.157072 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145956    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 21:16:00.157072 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145959    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 21:16:00.157072 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145962    4460 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 21:16:00.157072 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145965    4460 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 21:16:00.157072 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145968    4460 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 21:16:00.157072 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145970    4460 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 21:16:00.157072 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145974    4460 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 21:16:00.157072 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145977    4460 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 21:16:00.157072 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145980    4460 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 21:16:00.157072 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145983    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 21:16:00.157072 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145985    4460 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 21:16:00.157072 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145988    4460 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 21:16:00.157072 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145990    4460 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 21:16:00.157072 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145993    4460 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 21:16:00.157072 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145996    4460 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 21:16:00.157632 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.145999    4460 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 21:16:00.157632 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146001    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 21:16:00.157632 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146004    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 21:16:00.157632 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146006    4460 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 21:16:00.157632 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146009    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 21:16:00.157632 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146011    4460 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 21:16:00.157632 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146014    4460 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 21:16:00.157632 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146016    4460 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 21:16:00.157632 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146019    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 21:16:00.157632 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146021    4460 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 21:16:00.157632 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146024    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 21:16:00.157632 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146026    4460 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 21:16:00.157632 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146032    4460 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 21:16:00.157632 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146034    4460 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 21:16:00.157632 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146037    4460 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 21:16:00.157632 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.146063    4460 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 21:16:00.158302 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146105    4460 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 21:16:00.158302 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146109    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 21:16:00.158302 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146113    4460 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 21:16:00.158302 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146117    4460 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 21:16:00.158302 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146120    4460 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 21:16:00.158302 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146123    4460 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 21:16:00.158302 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146126    4460 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 21:16:00.158302 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146129    4460 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 21:16:00.158302 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146131    4460 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 21:16:00.158302 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146134    4460 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 21:16:00.158302 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146137    4460 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 21:16:00.158302 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146139    4460 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 21:16:00.158302 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146142    4460 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 21:16:00.158302 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146145    4460 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 21:16:00.158302 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146147    4460 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 21:16:00.158302 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146150    4460 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 21:16:00.158302 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146152    4460 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 21:16:00.158302 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146155    4460 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 21:16:00.158302 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146157    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 21:16:00.158302 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146160    4460 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 21:16:00.159135 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146162    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 21:16:00.159135 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146165    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 21:16:00.159135 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146167    4460 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 21:16:00.159135 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146170    4460 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 21:16:00.159135 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146173    4460 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 21:16:00.159135 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146175    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 21:16:00.159135 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146178    4460 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 21:16:00.159135 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146180    4460 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 21:16:00.159135 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146184    4460 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 21:16:00.159135 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146188    4460 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 21:16:00.159135 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146191    4460 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 21:16:00.159135 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146193    4460 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 21:16:00.159135 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146196    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 21:16:00.159135 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146199    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 21:16:00.159135 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146202    4460 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 21:16:00.159135 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146204    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 21:16:00.159135 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146207    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 21:16:00.159135 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146214    4460 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 21:16:00.159135 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146217    4460 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 21:16:00.159664 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146220    4460 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 21:16:00.159664 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146222    4460 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 21:16:00.159664 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146225    4460 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 21:16:00.159664 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146227    4460 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 21:16:00.159664 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146230    4460 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 21:16:00.159664 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146232    4460 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 21:16:00.159664 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146235    4460 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 21:16:00.159664 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146237    4460 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 21:16:00.159664 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146240    4460 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 21:16:00.159664 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146242    4460 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 21:16:00.159664 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146245    4460 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 21:16:00.159664 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146247    4460 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 21:16:00.159664 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146249    4460 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 21:16:00.159664 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146252    4460 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 21:16:00.159664 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146254    4460 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 21:16:00.159664 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146257    4460 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 21:16:00.159664 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146259    4460 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 21:16:00.159664 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146262    4460 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 21:16:00.159664 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146264    4460 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 21:16:00.160215 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146267    4460 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 21:16:00.160215 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146269    4460 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 21:16:00.160215 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146273    4460 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 21:16:00.160215 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146277    4460 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 21:16:00.160215 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146279    4460 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 21:16:00.160215 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146283    4460 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 21:16:00.160215 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146286    4460 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 21:16:00.160215 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146289    4460 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 21:16:00.160215 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146292    4460 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 21:16:00.160215 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146296    4460 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 21:16:00.160215 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146298    4460 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 21:16:00.160215 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146301    4460 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 21:16:00.160215 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146304    4460 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 21:16:00.160215 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146306    4460 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 21:16:00.160215 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146309    4460 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 21:16:00.160215 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146312    4460 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 21:16:00.160215 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146314    4460 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 21:16:00.160215 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146317    4460 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 21:16:00.160215 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146319    4460 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 21:16:00.160724 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146322    4460 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 21:16:00.160724 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146324    4460 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 21:16:00.160724 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146327    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 21:16:00.160724 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146329    4460 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 21:16:00.160724 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146332    4460 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 21:16:00.160724 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146334    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 21:16:00.160724 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146337    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 21:16:00.160724 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146339    4460 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 21:16:00.160724 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146342    4460 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 21:16:00.160724 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146344    4460 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 21:16:00.160724 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146347    4460 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 21:16:00.160724 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146350    4460 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 21:16:00.160724 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146354    4460 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 21:16:00.160724 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146357    4460 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 21:16:00.160724 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146359    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 21:16:00.160724 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146364    4460 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 21:16:00.160724 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146368    4460 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 21:16:00.160724 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146371    4460 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 21:16:00.160724 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146373    4460 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 21:16:00.160724 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146377    4460 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 21:16:00.161235 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146380    4460 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 21:16:00.161235 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146382    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 21:16:00.161235 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146385    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 21:16:00.161235 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146388    4460 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 21:16:00.161235 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146392    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 21:16:00.161235 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146394    4460 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 21:16:00.161235 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146398    4460 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 21:16:00.161235 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146400    4460 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 21:16:00.161235 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.146403    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 21:16:00.161235 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.151422    4460 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 31 21:16:00.161235 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.151439    4460 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 31 21:16:00.161235 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.151459    4460 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 31 21:16:00.161235 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.151464    4460 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 31 21:16:00.161235 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151524    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 21:16:00.161235 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151529    4460 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 21:16:00.161235 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151532    4460 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 21:16:00.161235 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151535    4460 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 21:16:00.161235 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151538    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 21:16:00.161235 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151541    4460 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 21:16:00.161707 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151543    4460 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 21:16:00.161707 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151546    4460 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 21:16:00.161707 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151549    4460 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 21:16:00.161707 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151552    4460 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 21:16:00.161707 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151554    4460 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 21:16:00.161707 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151557    4460 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 21:16:00.161707 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151559    4460 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 21:16:00.161707 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151562    4460 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 21:16:00.161707 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151565    4460 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 21:16:00.161707 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151568    4460 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 21:16:00.161707 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151571    4460 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 21:16:00.161707 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151574    4460 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 21:16:00.161707 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151577    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 21:16:00.161707 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151581    4460 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 21:16:00.161707 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151583    4460 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 21:16:00.161707 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151586    4460 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 21:16:00.161707 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151589    4460 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 21:16:00.161707 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151591    4460 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 21:16:00.161707 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151594    4460 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 21:16:00.161707 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151596    4460 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 21:16:00.162321 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151599    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 21:16:00.162321 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151602    4460 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 21:16:00.162321 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151605    4460 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 21:16:00.162321 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151607    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 21:16:00.162321 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151610    4460 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 21:16:00.162321 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151613    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 21:16:00.162321 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151615    4460 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 21:16:00.162321 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151618    4460 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 21:16:00.162321 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151620    4460 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 21:16:00.162321 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151623    4460 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 21:16:00.162321 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151626    4460 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 21:16:00.162321 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151628    4460 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 21:16:00.162321 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151631    4460 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 21:16:00.162321 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151633    4460 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 21:16:00.162321 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151636    4460 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 21:16:00.162321 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151638    4460 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 21:16:00.162321 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151641    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 21:16:00.162321 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151644    4460 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 21:16:00.162321 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151647    4460 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 21:16:00.162321 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151650    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 21:16:00.162804 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151653    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 21:16:00.162804 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151656    4460 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 21:16:00.162804 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151658    4460 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 21:16:00.162804 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151661    4460 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 21:16:00.162804 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151663    4460 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 21:16:00.162804 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151666    4460 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 21:16:00.162804 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151669    4460 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 21:16:00.162804 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151671    4460 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 21:16:00.162804 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151674    4460 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 21:16:00.162804 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151677    4460 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 21:16:00.162804 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151679    4460 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 21:16:00.162804 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151682    4460 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 21:16:00.162804 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151685    4460 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 21:16:00.162804 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151687    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 21:16:00.162804 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151690    4460 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 21:16:00.162804 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151693    4460 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 21:16:00.162804 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151695    4460 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 21:16:00.162804 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151698    4460 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 21:16:00.162804 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151701    4460 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 21:16:00.162804 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151703    4460 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 21:16:00.163335 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151706    4460 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 21:16:00.163335 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151709    4460 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 21:16:00.163335 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151711    4460 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 21:16:00.163335 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151714    4460 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 21:16:00.163335 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151717    4460 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 21:16:00.163335 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151720    4460 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 21:16:00.163335 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151722    4460 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 21:16:00.163335 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151725    4460 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 21:16:00.163335 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151728    4460 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 21:16:00.163335 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151731    4460 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 21:16:00.163335 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151735    4460 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 21:16:00.163335 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151739    4460 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 21:16:00.163335 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151742    4460 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 21:16:00.163335 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151745    4460 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 21:16:00.163335 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151747    4460 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 21:16:00.163335 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151750    4460 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 21:16:00.163335 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151753    4460 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 21:16:00.163335 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151755    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 21:16:00.163335 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151758    4460 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 21:16:00.163801 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151761    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 21:16:00.163801 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151763    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 21:16:00.163801 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151766    4460 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 21:16:00.163801 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151769    4460 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 21:16:00.163801 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151773    4460 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 21:16:00.163801 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151777    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 21:16:00.163801 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151780    4460 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 21:16:00.163801 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151783    4460 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 21:16:00.163801 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151786    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 21:16:00.163801 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151788    4460 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 21:16:00.163801 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151791    4460 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 21:16:00.163801 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151793    4460 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 21:16:00.163801 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151796    4460 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 21:16:00.163801 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151799    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 21:16:00.163801 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151801    4460 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 21:16:00.163801 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151804    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 21:16:00.163801 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151808    4460 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 21:16:00.163801 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151810    4460 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 21:16:00.163801 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151814    4460 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 21:16:00.164337 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151816    4460 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 21:16:00.164337 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.151819    4460 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 21:16:00.164337 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.151847    4460 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 21:16:00.164337 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152033    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 21:16:00.164337 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152039    4460 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 21:16:00.164337 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152042    4460 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 21:16:00.164337 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152045    4460 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 21:16:00.164337 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152049    4460 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 21:16:00.164337 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152054    4460 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 21:16:00.164337 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152057    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 21:16:00.164337 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152060    4460 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 21:16:00.164337 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152062    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 21:16:00.164337 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152065    4460 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 21:16:00.164337 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152069    4460 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 21:16:00.164337 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152073    4460 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 21:16:00.165050 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152076    4460 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 21:16:00.165050 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152080    4460 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 21:16:00.165050 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152082    4460 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 21:16:00.165050 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152085    4460 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 21:16:00.165050 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152088    4460 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 21:16:00.165050 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152090    4460 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 21:16:00.165050 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152093    4460 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 21:16:00.165050 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152096    4460 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 21:16:00.165050 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152098    4460 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 21:16:00.165050 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152101    4460 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 21:16:00.165050 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152104    4460 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 21:16:00.165050 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152106    4460 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 21:16:00.165050 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152109    4460 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 21:16:00.165050 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152112    4460 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 21:16:00.165050 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152114    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 21:16:00.165050 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152118    4460 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 21:16:00.165050 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152120    4460 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 21:16:00.165050 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152123    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 21:16:00.165050 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152126    4460 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 21:16:00.165050 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152128    4460 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 21:16:00.165661 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152131    4460 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 21:16:00.165661 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152133    4460 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 21:16:00.165661 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152136    4460 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 21:16:00.165661 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152138    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 21:16:00.165661 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152141    4460 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 21:16:00.165661 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152143    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 21:16:00.165661 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152147    4460 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 21:16:00.165661 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152149    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 21:16:00.165661 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152152    4460 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 21:16:00.165661 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152154    4460 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 21:16:00.165661 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152157    4460 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 21:16:00.165661 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152159    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 21:16:00.165661 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152162    4460 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 21:16:00.165661 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152165    4460 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 21:16:00.165661 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152167    4460 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 21:16:00.165661 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152170    4460 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 21:16:00.165661 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152172    4460 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 21:16:00.165661 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152175    4460 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 21:16:00.165661 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152177    4460 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 21:16:00.166154 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152180    4460 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 21:16:00.166154 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152183    4460 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 21:16:00.166154 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152186    4460 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 21:16:00.166154 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152188    4460 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 21:16:00.166154 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152191    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 21:16:00.166154 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152193    4460 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 21:16:00.166154 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152196    4460 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 21:16:00.166154 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152199    4460 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 21:16:00.166154 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152201    4460 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 21:16:00.166154 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152204    4460 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 21:16:00.166154 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152207    4460 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 21:16:00.166154 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152209    4460 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 21:16:00.166154 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152212    4460 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 21:16:00.166154 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152215    4460 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 21:16:00.166154 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152217    4460 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 21:16:00.166154 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152220    4460 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 21:16:00.166154 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152222    4460 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 21:16:00.166154 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152225    4460 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 21:16:00.166154 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152228    4460 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 21:16:00.166154 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152230    4460 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 21:16:00.166646 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152233    4460 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 21:16:00.166646 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152236    4460 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 21:16:00.166646 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152238    4460 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 21:16:00.166646 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152241    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 21:16:00.166646 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152243    4460 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 21:16:00.166646 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152246    4460 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 21:16:00.166646 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152248    4460 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 21:16:00.166646 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152252    4460 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 21:16:00.166646 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152255    4460 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 21:16:00.166646 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152257    4460 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 21:16:00.166646 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152260    4460 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 21:16:00.166646 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152262    4460 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 21:16:00.166646 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152265    4460 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 21:16:00.166646 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152267    4460 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 21:16:00.166646 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152270    4460 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 21:16:00.166646 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152272    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 21:16:00.166646 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152275    4460 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 21:16:00.166646 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152277    4460 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 21:16:00.166646 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152280    4460 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 21:16:00.167125 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152283    4460 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 21:16:00.167125 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152285    4460 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 21:16:00.167125 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152288    4460 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 21:16:00.167125 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152292    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 21:16:00.167125 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152296    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 21:16:00.167125 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152300    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 21:16:00.167125 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152304    4460 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 21:16:00.167125 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152307    4460 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 21:16:00.167125 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152311    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 21:16:00.167125 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152313    4460 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 21:16:00.167125 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152316    4460 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 21:16:00.167125 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152318    4460 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 21:16:00.167125 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152321    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 21:16:00.167125 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152324    4460 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 21:16:00.167125 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152327    4460 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 21:16:00.167125 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152330    4460 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 21:16:00.167507 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.152356    4460 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 21:16:00.167507 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152389    4460 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 21:16:00.167507 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152393    4460 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 21:16:00.167507 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152396    4460 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 21:16:00.167507 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152399    4460 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 21:16:00.167507 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152403    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 21:16:00.167507 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152406    4460 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 21:16:00.167507 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152408    4460 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 21:16:00.167507 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152412    4460 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 21:16:00.167507 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152415    4460 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 21:16:00.167507 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152417    4460 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 21:16:00.167507 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152420    4460 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 21:16:00.167507 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152423    4460 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 21:16:00.167507 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152425    4460 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 21:16:00.167507 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152428    4460 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 21:16:00.167507 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152430    4460 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 21:16:00.167909 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152433    4460 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 21:16:00.167909 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152436    4460 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 21:16:00.167909 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152438    4460 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 21:16:00.167909 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152441    4460 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 21:16:00.167909 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152443    4460 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 21:16:00.167909 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152446    4460 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 21:16:00.167909 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152448    4460 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 21:16:00.167909 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152451    4460 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 21:16:00.167909 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152453    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 21:16:00.167909 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152456    4460 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 21:16:00.167909 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152458    4460 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 21:16:00.167909 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152461    4460 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 21:16:00.167909 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152464    4460 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 21:16:00.167909 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152467    4460 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 21:16:00.167909 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152469    4460 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 21:16:00.167909 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152474    4460 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 21:16:00.167909 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152477    4460 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 21:16:00.167909 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152480    4460 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 21:16:00.167909 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152482    4460 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 21:16:00.167909 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152485    4460 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 21:16:00.168401 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152488    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 21:16:00.168401 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152490    4460 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 21:16:00.168401 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152493    4460 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 21:16:00.168401 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152495    4460 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 21:16:00.168401 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152498    4460 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 21:16:00.168401 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152500    4460 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 21:16:00.168401 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152503    4460 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 21:16:00.168401 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152505    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 21:16:00.168401 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152508    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 21:16:00.168401 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152511    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 21:16:00.168401 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152513    4460 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 21:16:00.168401 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152515    4460 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 21:16:00.168401 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152519    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 21:16:00.168401 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152521    4460 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 21:16:00.168401 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152524    4460 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 21:16:00.168401 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152527    4460 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 21:16:00.168401 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152530    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 21:16:00.168401 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152532    4460 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 21:16:00.168401 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152535    4460 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 21:16:00.168872 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152537    4460 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 21:16:00.168872 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152540    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 21:16:00.168872 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152542    4460 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 21:16:00.168872 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152545    4460 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 21:16:00.168872 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152547    4460 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 21:16:00.168872 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152551    4460 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 21:16:00.168872 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152555    4460 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 21:16:00.168872 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152558    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 21:16:00.168872 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152561    4460 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 21:16:00.168872 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152565    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 21:16:00.168872 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152567    4460 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 21:16:00.168872 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152571    4460 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 21:16:00.168872 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152575    4460 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 21:16:00.168872 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152578    4460 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 21:16:00.168872 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152580    4460 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 21:16:00.168872 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152583    4460 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 21:16:00.168872 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152585    4460 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 21:16:00.168872 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152588    4460 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 21:16:00.168872 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152591    4460 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 21:16:00.169469 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152594    4460 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 21:16:00.169469 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152596    4460 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 21:16:00.169469 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152599    4460 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 21:16:00.169469 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152602    4460 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 21:16:00.169469 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152605    4460 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 21:16:00.169469 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152607    4460 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 21:16:00.169469 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152611    4460 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 21:16:00.169469 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152615    4460 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 21:16:00.169469 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152618    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 21:16:00.169469 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152621    4460 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 21:16:00.169469 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152623    4460 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 21:16:00.169469 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152626    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 21:16:00.169469 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152628    4460 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 21:16:00.169469 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152631    4460 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 21:16:00.169469 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152633    4460 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 21:16:00.169469 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152636    4460 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 21:16:00.169469 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152638    4460 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 21:16:00.169469 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152641    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 21:16:00.169469 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152643    4460 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 21:16:00.169469 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152646    4460 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 21:16:00.170114 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152649    4460 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 21:16:00.170114 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152652    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 21:16:00.170114 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152655    4460 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 21:16:00.170114 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152658    4460 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 21:16:00.170114 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152661    4460 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 21:16:00.170114 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152663    4460 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 21:16:00.170114 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152666    4460 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 21:16:00.170114 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152668    4460 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 21:16:00.170114 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152671    4460 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 21:16:00.170114 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152673    4460 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 21:16:00.170114 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152676    4460 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 21:16:00.170114 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152678    4460 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 21:16:00.170114 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:00.152681    4460 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 21:16:00.170114 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.152733    4460 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 31 21:16:00.170114 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.153681    4460 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 31 21:16:00.170114 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.153734    4460 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 31 21:16:00.170114 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.153901    4460 server.go:1014] "Starting client certificate rotation"
Oct 31 21:16:00.170114 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.154048    4460 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 31 21:16:00.170580 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.154092    4460 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-30 21:10:20 +0000 UTC" deadline="2028-04-04 04:22:07.205583812 +0000 UTC"
Oct 31 21:16:00.170580 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.154127    4460 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12487h6m7.051459629s"
Oct 31 21:16:00.170580 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.154414    4460 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 21:16:00.170580 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.155466    4460 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 21:16:00.170580 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.157200    4460 log.go:25] "Validated CRI v1 runtime API"
Oct 31 21:16:00.170580 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.159193    4460 log.go:25] "Validated CRI v1 image API"
Oct 31 21:16:00.170580 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.159931    4460 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 31 21:16:00.170580 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.161372    4460 fs.go:135] Filesystem UUIDs: map[257b3f3f-5801-4157-8685-c44a82731d46:/dev/nvme0n1p4 5fee886e-9fef-4555-a7ff-01e4fa7789c5:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Oct 31 21:16:00.170923 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.161390    4460 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/18de7a0efc2046a20200fc40d809a3c10c031c1df3bf6a788bd52f6d80cb72b6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/18de7a0efc2046a20200fc40d809a3c10c031c1df3bf6a788bd52f6d80cb72b6/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1f17d26e4d3fcaecaf04ec9a71fde34b38a0fef0ad4d08e6d1361dde870d78d4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1f17d26e4d3fcaecaf04ec9a71fde34b38a0fef0ad4d08e6d1361dde870d78d4/userdata/shm major:0 minor:96 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2b851e26be51e365888088ee65bfb2f98feba4569e308ae07cc00b5b172091c4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2b851e26be51e365888088ee65bfb2f98feba4569e308ae07cc00b5b172091c4/userdata/shm major:0 minor:203 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/351a48ea03ea75f01303bf592add733800f0ce8ce400e0174218d6e3d26cb7e2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/351a48ea03ea75f01303bf592add733800f0ce8ce400e0174218d6e3d26cb7e2/userdata/shm major:0 minor:69 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3cd14d8f549c5518d91e7edb7842878da912dd7fbcb33d5d98cc2328eec1320b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3cd14d8f549c5518d91e7edb7842878da912dd7fbcb33d5d98cc2328eec1320b/userdata/shm major:0 minor:100 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/487ce8cd1e5f8b6ff2ebde6dd1c41b0ce1adf5472b52c672d688796311e56c99/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/487ce8cd1e5f8b6ff2ebde6dd1c41b0ce1adf5472b52c672d688796311e56c99/userdata/shm major:0 minor:215 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6d14dcafc9efa958880bd9dd2596c67f2ef87fa4a661a04445f36b4e4ac67382/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6d14dcafc9efa958880bd9dd2596c67f2ef87fa4a661a04445f36b4e4ac67382/userdata/shm major:0 minor:92 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/95adf62e87edd0f774f4919c151b796072147fad07c03e1766d1343358e0cd1e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/95adf62e87edd0f774f4919c151b796072147fad07c03e1766d1343358e0cd1e/userdata/shm major:0 minor:71 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9c2e2fda75e9ea6d6199dd288dc22f34b354dfe11647c8a42d67e72ac14c90fa/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9c2e2fda75e9ea6d6199dd288dc22f34b354dfe11647c8a42d67e72ac14c90fa/userdata/shm major:0 minor:217 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9eae0b17061554df6d278977b921196942e3eee88e7ee69f27ed07a44979b5eb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9eae0b17061554df6d278977b921196942e3eee88e7ee69f27ed07a44979b5eb/userdata/shm major:0 minor:88 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a1a687d9c746d237d1e4e3b121f6060ac051e41b732f5c7792cb1a49c504a9b3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a1a687d9c746d237d1e4e3b121f6060ac051e41b732f5c7792cb1a49c504a9b3/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a299c657ad84fdbb7a71f68209f2230e4ee9615f37cc5900c4b28c60c8f7f27f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a299c657ad84fdbb7a71f68209f2230e4ee9615f37cc5900c4b28c60c8f7f27f/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ba68cb8f841cddeb23c07da3a00ae9d65cf25b8e4a3b2e8ebdb6807d3e6dc8d7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ba68cb8f841cddeb23c07da3a00ae9d65cf25b8e4a3b2e8ebdb6807d3e6dc8d7/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f39f443d11426c0cd424f5ab150a5868364bc141077e7e513446918f3376f490/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f39f443d11426c0cd424f5ab150a5868364bc141077e7e513446918f3376f490/userdata/shm major:0 minor:224 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/08d86dfb-548b-4a83-a1c2-d6ce8d4a0091/volumes/kubernetes.io~projected/kube-api-access-pfq9g:{mountpoint:/var/lib/kubelet/pods/08d86dfb-548b-4a83-a1c2-d6ce8d4a0091/volumes/kubernetes.io~projected/kube-api-access-pfq9g major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0d542edf-e7a7-4898-bd81-17bd67e18003/volumes/kubernetes.io~projected/kube-api-access-lnrb2:{mountpoint:/var/lib/kubelet/pods/0d542edf-e7a7-4898-bd81-17bd67e18003/volumes/kubernetes.io~projected/kube-api-access-lnrb2 major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1695694b-12a1-4a6c-b931-bf7dda9ec50e/volumes/kubernetes.io~projected/kube-api-access-9pknd:{mountpoint:/var/lib/kubelet/pods/1695694b-12a1-4a6c-b931-bf7dda9ec50e/volumes/kubernetes.io~projected/kube-api-access-9pknd major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/22daf2b3-91ca-462b-8714-0e0b0ebbcb88/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/22daf2b3-91ca-462b-8714-0e0b0ebbcb88/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/23da9927-8255-4380-a3f0-b7ea9298ad40/volumes/kubernetes.io~projected/kube-api-access-ktdc9:{mountpoint:/var/lib/kubelet/pods/23da9927-8255-4380-a3f0-b7ea9298ad40/volumes/kubernetes.io~projected/kube-api-access-ktdc9 major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/26069598-5dc4-4c96-b55f-dd2f09884419/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/26069598-5dc4-4c96-b55f-dd2f09884419/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/26069598-5dc4-4c96-b55f-dd2f09884419/volumes/kubernetes.io~projected/kube-api-access-swpfh:{mountpoint:/var/lib/kubelet/pods/26069598-5dc4-4c96-b55f-dd2f09884419/volumes/kubernetes.io~projected/kube-api-access-swpfh major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/26069598-5dc4-4c96-b55f-dd2f09884419/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/26069598-5dc4-4c96-b55f-dd2f09884419/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3d9a2e4e-3a81-4ae6-80ed-68542e996259/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/3d9a2e4e-3a81-4ae6-80ed-68542e996259/volumes/kubernetes.io~secret/agent-certs major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3e254918-3cc1-4419-9820-37dff30f4445/volumes/kubernetes.io~projected/kube-api-access-mjb7n:{mountpoint:/var/lib/kubelet/pods/3e254918-3cc1-4419-9820-37dff30f4445/volumes/kubernetes.io~projected/kube-api-access-mjb7n major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5e12af35-fa4d-4c09-b428-a559b5ccc296/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/5e12af35-fa4d-4c09-b428-a559b5ccc296/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5e12af35-fa4d-4c09-b428-a559b5ccc296/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/5e12af35-fa4d-4c09-b428-a559b5ccc296/volumes/kubernetes.io~empty-dir/tmp major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5e12af35-fa4d-4c09-b428-a559b5ccc296/volumes/kubernetes.io~projected/kube-api-access-cbz7b:{mountpoint:/var/lib/kubelet/pods/5e12af35-fa4d-4c09-b428-a559b5ccc296/volumes/kubernetes.io~projected/kube-api-access-cbz7b major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6d63e78f-f474-4966-adb5-401be54f4ab4/volumes/kubernetes.io~projected/kube-api-access-97qrk:{mountpoint:/var/lib/kubelet/pods/6d63e78f-f474-4966-adb5-401be54f4ab4/volumes/kubernetes.io~projected/kube-api-access-97qrk major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6d63e78f-f474-4966-adb5-401be54f4ab4/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/6d63e78f-f474-4966-adb5-401be54f4ab4/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/91776cb8-cbd4-441a-8d4a-17e342717812/volumes/kubernetes.io~projected/kube-api-access-mb6ph:{mountpoint:/var/lib/kubelet/pods/91776cb8-cbd4-441a-8d4a-17e342717812/volumes/kubernetes.io~projected/kube-api-access-mb6ph major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/94b754a2-a8f3-494c-89bb-e903f9c2cdb0/volumes/kubernetes.io~projected/kube-api-access-c66dd:{mountpoint:/var/lib/kubelet/pods/94b754a2-a8f3-494c-89bb-e903f9c2cdb0/volumes/kubernetes.io~projected/kube-api-access-c66dd major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a6207663-1cfa-48d9-8382-12973bbf29fd/volumes/kubernetes.io~projected/kube-api-access-mnxs9:{mountpoint:/var/lib/kubelet/pods/a6207663-1cfa-48d9-8382-12973bbf29fd/volumes/kubernetes.io~projected/kube-api-access-mnxs9 major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a6207663-1cfa-48d9-8382-12973bbf29fd/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/a6207663-1cfa-48d9-8382-12973bbf29fd/volumes/kubernetes.io~secret/metrics-tls major:0 minor:223 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b65919f8-6cc9-4061-8a67-e1d5455c596d/volumes/kubernetes.io~projected/kube-api-access-pk72s:{mountpoint:/var/lib/kubelet/pods/b65919f8-6cc9-4061-8a67-e1d5455c596d/volumes/kubernetes.io~projected/kube-api-access-pk72s major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bb42ddd8-d246-4bf3-ae35-27aa4584b163/volumes/kubernetes.io~projected/kube-api-access-2rh6z:{mountpoint:/var/lib/kubelet/pods/bb42ddd8-d246-4bf3-ae35-27aa4584b163/volumes/kubernetes.io~projected/kube-api-access-2rh6z major:0 minor:63 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/47fa64ce0588d2e12bf432cc46aa9b1c57031506fcdbeaa2e5f019d7dee5c653/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/b290f1f67b7ab20ca63426517901c653ed3b6ebb68e4a08f4714bfd33160b8e1/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/f92626888645eec9a0b72c3e1a786e989075c7122483b5411af8636afb2a312e/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/3ecd66f1a203800f217dda7647e857a82dbc8e37e944609be39168316fc0cb1b/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/5d935e3698025cdb7a1c0fc2ae2dc0d29bf0dff0b104f9c32dc436328ffd8458/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/419ddc0e95bd4538a0812d5f19aae5aca094612ea53063c9680f64a652d87c4f/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/5631f0ef0c06588d3e6f9277a30e717fb1f1c1a978225aca0dbc32756a2f212f/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/e86bcd82a3bb91259623fad8ceafb1af31381b0bc06a4ed1493e87566cd37cc7/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-133:{mountpoint:/var/lib/containers/storage/overlay/740a02b9e8494e46643e81c8ca793de8677ec26cd3409edf86e7ac0550ba820d/merged major:0 minor:133 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/d6ad172f4505661b0e5560161bd18205b8e6a6af1553a4730fd9015ed9751dfb/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-143:{mountpoint:/var/lib/containers/storage/overlay/3c3c8b9b0316bc9b719c17750ba414c2dc7b45bf2e5c296a99d21f29ad0b0f2b/merged major:0 minor:143 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/adc5ff1c9afe883e189a318562db433b040e7c6b7bce33a36deb4779c74c1e85/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-154:{mountpoint:/var/lib/containers/storage/overlay/bdaacc765514e201d9d37d13bb8f64615efd27f51e0f2d4e43b31e6d2fb6c0fb/merged major:0 minor:154 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/d0e6535539097c616d85073bd647c08a9b3cc68aafe6cb998472f7952dc33fb2/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/851cb916e6857d7ffe6bb4a1cfc13b18db19a952eb2070e3dea509ebdb705c2e/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/2d79a1e0b065de9c1857b331c9d49efe241c1ff7125c69c572bd55985091f4bf/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/978b0e1eff9e13c1ca20c52e73b89c98e40d03d4a17a0885a26f8a28883a6156/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/19e0962e1d33d5d872f37ed8f9fa35d4b2df79c3888f06b6d90f151eeef0252b/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/9c380ccc490fe2f82028d72791f7d47450d2bcb365c91ebcc3221b5ef1aa6f85/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/86194824d7664e5f2a62b1d4d80ffdb8d73cda948520b0c4f1900874999fce1a/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-205:{mountpoint:/var/lib/containers/storage/overlay/b9c67b292a565e6919efe27c6e625adfa4e68e3605a107632645d1e31a3b6d77/merged major:0 minor:205 fsType:overlay blockSize:0} overlay_0-207:{mountpoint:/var/lib/containers/storage/overlay/e61cbca9eabf17fccb7b30d4f89dae1a5f0875399c9002430b3ff368f33621df/merged major:0 minor:207 fsType:overlay blockSize:0} overlay_0-219:{mountpoint:/var/lib/containers/storage/overlay/a4dba50f0bb1c17f5fdb484f9f422df5da94d3a33710f350c35c25e28ab36ccc/merged major:0 minor:219 fsType:overlay blockSize:0} overlay_0-221:{mountpoint:/var/lib/containers/storage/overlay/d934850138a5bd1e19fa0a2d8687347b30dfc47ea95ab69d4c68ef6de48bb99e/merged major:0 minor:221 fsType:overlay blockSize:0} overlay_0-226:{mountpoint:/var/lib/containers/storage/overlay/36b47026b241099ffb93e53ec3996a02eb53100aed7431a7b8f267fff6652e92/merged major:0 minor:226 fsType:overlay blockSize:0} overlay_0-228:{mountpoint:/var/lib/containers/storage/overlay/f287cb7a32d4144e85e8abc89cb416d88d657308c834f20a62f1076371220d1b/merged major:0 minor:228 fsType:overlay blockSize:0} overlay_0-230:{mountpoint:/var/lib/containers/storage/overlay/84a325e8af07b9f444799542ea6991eef526e814295ff43c6f44af84b5c37991/merged major:0 minor:230 fsType:overlay blockSize:0} overlay_0-236:{mountpoint:/var/lib/containers/storage/overlay/dd499fc5d59b0208cb042e58e2a4b568035fa3caecf7b60781edc654948a5848/merged major:0 minor:236 fsType:overlay blockSize:0} overlay_0-238:{mountpoint:/var/lib/containers/storage/overlay/333e472f9220dd1279beeffc3da085c60a5a9ae938bfe18b3e2a1f153395e8a2/merged major:0 minor:238 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/235288c492eb88ede68381e17e2b7defa09efe7f0a416ac69b5ba461761c9067/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/2e031f7124d0951738bb58c500233a89bbdec4bf6205a9349d85ee36d1ea8f8b/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/1d7abc47ff6e653c841b49ac5e6e96a4fdb21acef26c1a34bbd50da50d8c2c0a/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-75:{mountpoint:/var/lib/containers/storage/overlay/e29762fdbccefc67faffe341e2fb855cacddaf5769a7fb46c26d6e39656c935b/merged major:0 minor:75 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/5a266ad30b64921133ca54364c080aaf02e828986667a89a25a974ce7176a18d/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/ca7d6e32267d197a37d761a6b9891742c10603ae53cbe12107337befc41c9b8f/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/5316b98e8287448d5cf405402eeeb7f45e60cbbe6b7e9d77e27ee44eec4a08a4/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/06610489da448474749b367e03017bffe11481b1318db7be8c1940a5f9f49b7d/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-90:{mountpoint:/var/lib/containers/storage/overlay/df6acd3c8eff03bf61534e2191b8e5a020892fb030536fc8d34d53a97506c41e/merged major:0 minor:90 fsType:overlay blockSize:0} overlay_0-94:{mountpoint:/var/lib/containers/storage/overlay/59b92829eecd76b0f697eecd53b062a63c93a77d05a48ccfe9224a3e119e3ea5/merged major:0 minor:94 fsType:overlay blockSize:0} overlay_0-98:{mountpoint:/var/lib/containers/storage/overlay/e6a2c05e5cb6a55070b63d245213ad880dd607d37f73809208d2d2b309d22a80/merged major:0 minor:98 fsType:overlay blockSize:0}]
Oct 31 21:16:00.171240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.168337    4460 manager.go:217] Machine: {Timestamp:2026-10-31 21:16:00.167886014 +0000 UTC m=+0.084195434 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098455 MemoryCapacity:33164333056 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec221116a41121a82aa1d46fe2bf9f96 SystemUUID:ec221116-a411-21a8-2aa1-d46fe2bf9f96 BootID:c4974a24-3e9e-421a-af93-99a9bf8c0cca Filesystems:[{Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/22daf2b3-91ca-462b-8714-0e0b0ebbcb88/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:209 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-236 DeviceMajor:0 DeviceMinor:236 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1695694b-12a1-4a6c-b931-bf7dda9ec50e/volumes/kubernetes.io~projected/kube-api-access-9pknd DeviceMajor:0 DeviceMinor:201 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-221 DeviceMajor:0 DeviceMinor:221 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-154 DeviceMajor:0 DeviceMinor:154 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3d9a2e4e-3a81-4ae6-80ed-68542e996259/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:58 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/91776cb8-cbd4-441a-8d4a-17e342717812/volumes/kubernetes.io~projected/kube-api-access-mb6ph DeviceMajor:0 DeviceMinor:64 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-90 DeviceMajor:0 DeviceMinor:90 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6d63e78f-f474-4966-adb5-401be54f4ab4/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2b851e26be51e365888088ee65bfb2f98feba4569e308ae07cc00b5b172091c4/userdata/shm DeviceMajor:0 DeviceMinor:203 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/08d86dfb-548b-4a83-a1c2-d6ce8d4a0091/volumes/kubernetes.io~projected/kube-api-access-pfq9g DeviceMajor:0 DeviceMinor:214 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-238 DeviceMajor:0 DeviceMinor:238 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-133 DeviceMajor:0 DeviceMinor:133 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/351a48ea03ea75f01303bf592add733800f0ce8ce400e0174218d6e3d26cb7e2/userdata/shm DeviceMajor:0 DeviceMinor:69 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/95adf62e87edd0f774f4919c151b796072147fad07c03e1766d1343358e0cd1e/userdata/shm DeviceMajor:0 DeviceMinor:71 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-75 DeviceMajor:0 DeviceMinor:75 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-94 DeviceMajor:0 DeviceMinor:94 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-143 DeviceMajor:0 DeviceMinor:143 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-219 DeviceMajor:0 DeviceMinor:219 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/94b754a2-a8f3-494c-89bb-e903f9c2cdb0/volumes/kubernetes.io~projected/kube-api-access-c66dd DeviceMajor:0 DeviceMinor:66 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-226 DeviceMajor:0 DeviceMinor:226 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a1a687d9c746d237d1e4e3b121f6060ac051e41b732f5c7792cb1a49c504a9b3/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/b65919f8-6cc9-4061-8a67-e1d5455c596d/volumes/kubernetes.io~projected/kube-api-access-pk72s DeviceMajor:0 DeviceMinor:60 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/23da9927-8255-4380-a3f0-b7ea9298ad40/volumes/kubernetes.io~projected/kube-api-access-ktdc9 DeviceMajor:0 DeviceMinor:61 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6d63e78f-f474-4966-adb5-401be54f4ab4/volumes/kubernetes.io~projected/kube-api-access-97qrk DeviceMajor:0 DeviceMinor:202 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/0d542edf-e7a7-4898-bd81-17bd67e18003/volumes/kubernetes.io~projected/kube-api-access-lnrb2 DeviceMajor:0 DeviceMinor:62 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6d14dcafc9efa958880bd9dd2596c67f2ef87fa4a661a04445f36b4e4ac67382/userdata/shm DeviceMajor:0 DeviceMinor:92 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a6207663-1cfa-48d9-8382-12973bbf29fd/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:223 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/5e12af35-fa4d-4c09-b428-a559b5ccc296/volumes/kubernetes.io~projected/kube-api-access-cbz7b DeviceMajor:0 DeviceMinor:65 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a299c657ad84fdbb7a71f68209f2230e4ee9615f37cc5900c4b28c60c8f7f27f/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-207 DeviceMajor:0 DeviceMinor:207 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582164480 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1f17d26e4d3fcaecaf04ec9a71fde34b38a0fef0ad4d08e6d1361dde870d78d4/userdata/shm DeviceMajor:0 DeviceMinor:96 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-205 DeviceMajor:0 DeviceMinor:205 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/487ce8cd1e5f8b6ff2ebde6dd1c41b0ce1adf5472b52c672d688796311e56c99/userdata/shm DeviceMajor:0 DeviceMinor:215 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ba68cb8f841cddeb23c07da3a00ae9d65cf25b8e4a3b2e8ebdb6807d3e6dc8d7/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-98 DeviceMajor:0 DeviceMinor:98 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/26069598-5dc4-4c96-b55f-dd2f09884419/volumes/kubernetes.io~projected/kube-api-access-swpfh DeviceMajor:0 DeviceMinor:67 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/26069598-5dc4-4c96-b55f-dd2f09884419/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9c2e2fda75e9ea6d6199dd288dc22f34b354dfe11647c8a42d67e72ac14c90fa/userdata/shm DeviceMajor:0 DeviceMinor:217 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-230 DeviceMajor:0 DeviceMinor:230 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9eae0b17061554df6d278977b921196942e3eee88e7ee69f27ed07a44979b5eb/userdata/shm DeviceMajor:0 DeviceMinor:88 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a6207663-1cfa-48d9-8382-12973bbf29fd/volumes/kubernetes.io~projected/kube-api-access-mnxs9 DeviceMajor:0 DeviceMinor:197 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/3e254918-3cc1-4419-9820-37dff30f4445/volumes/kubernetes.io~projected/kube-api-access-mjb7n DeviceMajor:0 DeviceMinor:59 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/18de7a0efc2046a20200fc40d809a3c10c031c1df3bf6a788bd52f6d80cb72b6/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3cd14d8f549c5518d91e7edb7842878da912dd7fbcb33d5d98cc2328eec1320b/userdata/shm DeviceMajor:0 DeviceMinor:100 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f39f443d11426c0cd424f5ab150a5868364bc141077e7e513446918f3376f490/userdata/shm DeviceMajor:0 DeviceMinor:224 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/26069598-5dc4-4c96-b55f-dd2f09884419/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:55 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/5e12af35-fa4d-4c09-b428-a559b5ccc296/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:56 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/5e12af35-fa4d-4c09-b428-a559b5ccc296/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:57 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/bb42ddd8-d246-4bf3-ae35-27aa4584b163/volumes/kubernetes.io~projected/kube-api-access-2rh6z DeviceMajor:0 DeviceMinor:63 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-228 DeviceMajor:0 DeviceMinor:228 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:2b851e26be51e36 MacAddress:2e:0e:79:71:93:53 Speed:10000 Mtu:8901} {Name:487ce8cd1e5f8b6 MacAddress:c6:78:86:e2:5e:e5 Speed:10000 Mtu:8901} {Name:9c2e2fda75e9ea6 MacAddress:fa:00:3b:fa:2b:8f Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:e0:b1:f1:3f Speed:0 Mtu:9001} {Name:br-int MacAddress:4e:0a:41:22:ab:92 Speed:0 Mtu:8901} {Name:ens5 MacAddress:02:ff:e0:b1:f1:3f Speed:0 Mtu:9001} {Name:f39f443d11426c0 MacAddress:8a:82:76:8d:a1:67 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:ca:7d:c9:5b:10:48 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:f6:2a:a0:38:ef:8d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164333056 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 31 21:16:00.171240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.168578    4460 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 31 21:16:00.171240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.168617    4460 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 31 21:16:00.171240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.168917    4460 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 31 21:16:00.171240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.168955    4460 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-157.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 31 21:16:00.171240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.169080    4460 topology_manager.go:143] "Creating topology manager with none policy"
Oct 31 21:16:00.171240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.169089    4460 container_manager_linux.go:311] "Creating device plugin manager"
Oct 31 21:16:00.171240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.169099    4460 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 21:16:00.171240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.169117    4460 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 21:16:00.171240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.169202    4460 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 31 21:16:00.171240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.169362    4460 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 31 21:16:00.171240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.169470    4460 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 31 21:16:00.171240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.169505    4460 kubelet.go:493] "Attempting to sync node with API server"
Oct 31 21:16:00.171240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.169519    4460 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 31 21:16:00.171240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.169529    4460 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 31 21:16:00.171240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.169537    4460 kubelet.go:405] "Adding apiserver pod source"
Oct 31 21:16:00.171240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.169545    4460 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 31 21:16:00.171240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.170160    4460 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 31 21:16:00.171240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.170170    4460 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 21:16:00.171240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.170755    4460 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 31 21:16:00.173319 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.173301    4460 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 31 21:16:00.173319 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.173318    4460 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 21:16:00.173453 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.173433    4460 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 31 21:16:00.173656 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.173640    4460 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 31 21:16:00.173724 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.173680    4460 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 31 21:16:00.173802 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.173791    4460 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 31 21:16:00.173855 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.173811    4460 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 31 21:16:00.173855 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.173821    4460 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 31 21:16:00.173855 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.173831    4460 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 31 21:16:00.173855 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.173840    4460 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 31 21:16:00.173855 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.173848    4460 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 31 21:16:00.174114 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.173858    4460 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 31 21:16:00.174114 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.173868    4460 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 31 21:16:00.174114 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.173881    4460 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 31 21:16:00.174114 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.173890    4460 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 31 21:16:00.174114 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.173909    4460 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 31 21:16:00.174114 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.173923    4460 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 31 21:16:00.174114 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.173975    4460 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 31 21:16:00.174114 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.173987    4460 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 31 21:16:00.177259 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.177243    4460 server.go:1263] "Started kubelet"
Oct 31 21:16:00.177369 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.177308    4460 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 31 21:16:00.177431 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.177362    4460 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 31 21:16:00.177472 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.177456    4460 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 31 21:16:00.177719 ip-10-0-136-157 systemd[1]: Started Kubernetes Kubelet.
Oct 31 21:16:00.177854 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.177749    4460 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 31 21:16:00.178571 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.178489    4460 server.go:317] "Adding debug handlers to kubelet server"
Oct 31 21:16:00.181555 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.180569    4460 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 31 21:16:00.181555 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.180597    4460 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 31 21:16:00.181555 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.180782    4460 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 31 21:16:00.181555 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.180796    4460 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 31 21:16:00.181555 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.180911    4460 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 31 21:16:00.181555 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:16:00.180972    4460 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-157.ec2.internal\" not found"
Oct 31 21:16:00.181555 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.181181    4460 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-30 21:10:22 +0000 UTC" deadline="2028-04-18 05:30:21.334800862 +0000 UTC"
Oct 31 21:16:00.181555 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.181220    4460 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12824h14m21.153585158s"
Oct 31 21:16:00.183322 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.183276    4460 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 31 21:16:00.183322 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.183310    4460 factory.go:55] Registering systemd factory
Oct 31 21:16:00.183322 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.183319    4460 factory.go:223] Registration of the systemd container factory successfully
Oct 31 21:16:00.184568 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.184524    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0d542edf-e7a7-4898-bd81-17bd67e18003" volumeName="kubernetes.io/configmap/0d542edf-e7a7-4898-bd81-17bd67e18003-iptables-alerter-script" seLinuxMountContext=""
Oct 31 21:16:00.184655 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.184569    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5e12af35-fa4d-4c09-b428-a559b5ccc296" volumeName="kubernetes.io/empty-dir/5e12af35-fa4d-4c09-b428-a559b5ccc296-tmp" seLinuxMountContext=""
Oct 31 21:16:00.184655 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.184584    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a6207663-1cfa-48d9-8382-12973bbf29fd" volumeName="kubernetes.io/projected/a6207663-1cfa-48d9-8382-12973bbf29fd-kube-api-access-mnxs9" seLinuxMountContext=""
Oct 31 21:16:00.184655 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.184602    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bb42ddd8-d246-4bf3-ae35-27aa4584b163" volumeName="kubernetes.io/projected/bb42ddd8-d246-4bf3-ae35-27aa4584b163-kube-api-access-2rh6z" seLinuxMountContext=""
Oct 31 21:16:00.184655 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.184615    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="23da9927-8255-4380-a3f0-b7ea9298ad40" volumeName="kubernetes.io/projected/23da9927-8255-4380-a3f0-b7ea9298ad40-kube-api-access-ktdc9" seLinuxMountContext=""
Oct 31 21:16:00.184655 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.184632    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="26069598-5dc4-4c96-b55f-dd2f09884419" volumeName="kubernetes.io/secret/26069598-5dc4-4c96-b55f-dd2f09884419-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 31 21:16:00.184655 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.184647    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5e12af35-fa4d-4c09-b428-a559b5ccc296" volumeName="kubernetes.io/projected/5e12af35-fa4d-4c09-b428-a559b5ccc296-kube-api-access-cbz7b" seLinuxMountContext=""
Oct 31 21:16:00.184972 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.184660    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d63e78f-f474-4966-adb5-401be54f4ab4" volumeName="kubernetes.io/secret/6d63e78f-f474-4966-adb5-401be54f4ab4-cert" seLinuxMountContext=""
Oct 31 21:16:00.184972 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.184805    4460 factory.go:153] Registering CRI-O factory
Oct 31 21:16:00.184972 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.184819    4460 factory.go:223] Registration of the crio container factory successfully
Oct 31 21:16:00.184972 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.184838    4460 factory.go:103] Registering Raw factory
Oct 31 21:16:00.184972 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.184853    4460 manager.go:1199] Started watching for new ooms in manager
Oct 31 21:16:00.185221 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.184914    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="91776cb8-cbd4-441a-8d4a-17e342717812" volumeName="kubernetes.io/configmap/91776cb8-cbd4-441a-8d4a-17e342717812-cni-binary-copy" seLinuxMountContext=""
Oct 31 21:16:00.185221 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.185144    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="91776cb8-cbd4-441a-8d4a-17e342717812" volumeName="kubernetes.io/projected/91776cb8-cbd4-441a-8d4a-17e342717812-kube-api-access-mb6ph" seLinuxMountContext=""
Oct 31 21:16:00.185221 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.185175    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a6207663-1cfa-48d9-8382-12973bbf29fd" volumeName="kubernetes.io/empty-dir/a6207663-1cfa-48d9-8382-12973bbf29fd-tmp-dir" seLinuxMountContext=""
Oct 31 21:16:00.185221 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.185210    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bb42ddd8-d246-4bf3-ae35-27aa4584b163" volumeName="kubernetes.io/empty-dir/bb42ddd8-d246-4bf3-ae35-27aa4584b163-tmp-dir" seLinuxMountContext=""
Oct 31 21:16:00.185402 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.185229    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="23da9927-8255-4380-a3f0-b7ea9298ad40" volumeName="kubernetes.io/configmap/23da9927-8255-4380-a3f0-b7ea9298ad40-serviceca" seLinuxMountContext=""
Oct 31 21:16:00.185402 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.185249    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3d9a2e4e-3a81-4ae6-80ed-68542e996259" volumeName="kubernetes.io/configmap/3d9a2e4e-3a81-4ae6-80ed-68542e996259-konnectivity-ca" seLinuxMountContext=""
Oct 31 21:16:00.185402 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.185288    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3d9a2e4e-3a81-4ae6-80ed-68542e996259" volumeName="kubernetes.io/secret/3d9a2e4e-3a81-4ae6-80ed-68542e996259-agent-certs" seLinuxMountContext=""
Oct 31 21:16:00.185402 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.185308    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d63e78f-f474-4966-adb5-401be54f4ab4" volumeName="kubernetes.io/projected/6d63e78f-f474-4966-adb5-401be54f4ab4-kube-api-access-97qrk" seLinuxMountContext=""
Oct 31 21:16:00.185402 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.185320    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94b754a2-a8f3-494c-89bb-e903f9c2cdb0" volumeName="kubernetes.io/configmap/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 31 21:16:00.185402 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.185357    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a6207663-1cfa-48d9-8382-12973bbf29fd" volumeName="kubernetes.io/secret/a6207663-1cfa-48d9-8382-12973bbf29fd-metrics-tls" seLinuxMountContext=""
Oct 31 21:16:00.185402 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.185373    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0d542edf-e7a7-4898-bd81-17bd67e18003" volumeName="kubernetes.io/projected/0d542edf-e7a7-4898-bd81-17bd67e18003-kube-api-access-lnrb2" seLinuxMountContext=""
Oct 31 21:16:00.185402 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.185384    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1695694b-12a1-4a6c-b931-bf7dda9ec50e" volumeName="kubernetes.io/projected/1695694b-12a1-4a6c-b931-bf7dda9ec50e-kube-api-access-9pknd" seLinuxMountContext=""
Oct 31 21:16:00.185751 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.185406    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94b754a2-a8f3-494c-89bb-e903f9c2cdb0" volumeName="kubernetes.io/configmap/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-cni-binary-copy" seLinuxMountContext=""
Oct 31 21:16:00.185751 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.185438    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b65919f8-6cc9-4061-8a67-e1d5455c596d" volumeName="kubernetes.io/projected/b65919f8-6cc9-4061-8a67-e1d5455c596d-kube-api-access-pk72s" seLinuxMountContext=""
Oct 31 21:16:00.185751 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.185461    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="08d86dfb-548b-4a83-a1c2-d6ce8d4a0091" volumeName="kubernetes.io/projected/08d86dfb-548b-4a83-a1c2-d6ce8d4a0091-kube-api-access-pfq9g" seLinuxMountContext=""
Oct 31 21:16:00.185751 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.185474    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1695694b-12a1-4a6c-b931-bf7dda9ec50e" volumeName="kubernetes.io/configmap/1695694b-12a1-4a6c-b931-bf7dda9ec50e-kube-rbac-proxy-cm" seLinuxMountContext=""
Oct 31 21:16:00.185751 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.185477    4460 manager.go:319] Starting recovery of all containers
Oct 31 21:16:00.185751 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.185485    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1695694b-12a1-4a6c-b931-bf7dda9ec50e" volumeName="kubernetes.io/empty-dir/1695694b-12a1-4a6c-b931-bf7dda9ec50e-data-volume" seLinuxMountContext=""
Oct 31 21:16:00.185751 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.185521    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5e12af35-fa4d-4c09-b428-a559b5ccc296" volumeName="kubernetes.io/empty-dir/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-tuned" seLinuxMountContext=""
Oct 31 21:16:00.185751 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.185536    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94b754a2-a8f3-494c-89bb-e903f9c2cdb0" volumeName="kubernetes.io/configmap/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 31 21:16:00.185751 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.185553    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a6207663-1cfa-48d9-8382-12973bbf29fd" volumeName="kubernetes.io/configmap/a6207663-1cfa-48d9-8382-12973bbf29fd-config-volume" seLinuxMountContext=""
Oct 31 21:16:00.185751 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.185613    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="26069598-5dc4-4c96-b55f-dd2f09884419" volumeName="kubernetes.io/configmap/26069598-5dc4-4c96-b55f-dd2f09884419-env-overrides" seLinuxMountContext=""
Oct 31 21:16:00.185751 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.185627    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="22daf2b3-91ca-462b-8714-0e0b0ebbcb88" volumeName="kubernetes.io/secret/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-original-pull-secret" seLinuxMountContext=""
Oct 31 21:16:00.185751 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.185649    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="26069598-5dc4-4c96-b55f-dd2f09884419" volumeName="kubernetes.io/projected/26069598-5dc4-4c96-b55f-dd2f09884419-kube-api-access-swpfh" seLinuxMountContext=""
Oct 31 21:16:00.185751 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.185663    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94b754a2-a8f3-494c-89bb-e903f9c2cdb0" volumeName="kubernetes.io/projected/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-kube-api-access-c66dd" seLinuxMountContext=""
Oct 31 21:16:00.185751 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.185678    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="26069598-5dc4-4c96-b55f-dd2f09884419" volumeName="kubernetes.io/configmap/26069598-5dc4-4c96-b55f-dd2f09884419-ovnkube-config" seLinuxMountContext=""
Oct 31 21:16:00.185751 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.185689    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="26069598-5dc4-4c96-b55f-dd2f09884419" volumeName="kubernetes.io/configmap/26069598-5dc4-4c96-b55f-dd2f09884419-ovnkube-script-lib" seLinuxMountContext=""
Oct 31 21:16:00.185751 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.185698    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3e254918-3cc1-4419-9820-37dff30f4445" volumeName="kubernetes.io/projected/3e254918-3cc1-4419-9820-37dff30f4445-kube-api-access-mjb7n" seLinuxMountContext=""
Oct 31 21:16:00.185751 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.185708    4460 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="91776cb8-cbd4-441a-8d4a-17e342717812" volumeName="kubernetes.io/configmap/91776cb8-cbd4-441a-8d4a-17e342717812-multus-daemon-config" seLinuxMountContext=""
Oct 31 21:16:00.185751 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.185718    4460 reconstruct.go:97] "Volume reconstruction finished"
Oct 31 21:16:00.185751 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.185724    4460 reconciler.go:29] "Reconciler: start to sync state"
Oct 31 21:16:00.196808 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.196786    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ssl9_26069598-5dc4-4c96-b55f-dd2f09884419/ovn-acl-logging/0.log"
Oct 31 21:16:00.199074 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.199053    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ssl9_26069598-5dc4-4c96-b55f-dd2f09884419/ovn-acl-logging/0.log"
Oct 31 21:16:00.200303 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.200282    4460 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 31 21:16:00.216652 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.216628    4460 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 31 21:16:00.216652 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.216652    4460 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 31 21:16:00.216852 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.216669    4460 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 31 21:16:00.216852 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:16:00.216712    4460 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 31 21:16:00.229047 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.229022    4460 generic.go:356] "Generic (PLEG): container finished" podID="94b754a2-a8f3-494c-89bb-e903f9c2cdb0" containerID="4045d9f0e5611a94da16771cb969f6c7634f11d0c51f7b4c580f234db6d6025b" exitCode=0
Oct 31 21:16:00.229047 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.229044    4460 generic.go:356] "Generic (PLEG): container finished" podID="94b754a2-a8f3-494c-89bb-e903f9c2cdb0" containerID="3fd1811be54ec6d2a66b3ec47973cacf08b0987bcdb4f42a6405fba6dbcfbd09" exitCode=0
Oct 31 21:16:00.229047 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.229050    4460 generic.go:356] "Generic (PLEG): container finished" podID="94b754a2-a8f3-494c-89bb-e903f9c2cdb0" containerID="156922c32971703eb14808251a2d6138e7db58f1e223b0a6712ac65c34cec560" exitCode=0
Oct 31 21:16:00.229242 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.229056    4460 generic.go:356] "Generic (PLEG): container finished" podID="94b754a2-a8f3-494c-89bb-e903f9c2cdb0" containerID="9e1b2a5e32243f0fc6f91f1d728d73f95974afb8cde980bab666cf82e12551a2" exitCode=0
Oct 31 21:16:00.229242 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.229061    4460 generic.go:356] "Generic (PLEG): container finished" podID="94b754a2-a8f3-494c-89bb-e903f9c2cdb0" containerID="417a0858cb05b0a4cf00fbbf61a0e0501c2e70ae585856b52c0b8963ab69afec" exitCode=0
Oct 31 21:16:00.229242 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.229066    4460 generic.go:356] "Generic (PLEG): container finished" podID="94b754a2-a8f3-494c-89bb-e903f9c2cdb0" containerID="2bc21d8c997cbc37ae094c37e1e7e8ccb4879179b97610b353172f235fae11cf" exitCode=0
Oct 31 21:16:00.232847 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.232820    4460 generic.go:356] "Generic (PLEG): container finished" podID="050c66d67c3ed1a5b46b1f7ad6044575" containerID="52dab86d69c25d3fe6273ec3333d4be9e53e40aedf149a68ff3ea8917e0fcc99" exitCode=0
Oct 31 21:16:00.238389 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.238353    4460 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 21:16:00.238496 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.238475    4460 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 21:16:00.238560 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.238524    4460 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 31 21:16:00.248912 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.248895    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ssl9_26069598-5dc4-4c96-b55f-dd2f09884419/ovn-acl-logging/0.log"
Oct 31 21:16:00.249243 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.249223    4460 generic.go:356] "Generic (PLEG): container finished" podID="26069598-5dc4-4c96-b55f-dd2f09884419" containerID="f195879d5c7baf5c73f623c2c35682fb10da49088b7582a3fc72f326b9a8f67a" exitCode=1
Oct 31 21:16:00.252390 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.252369    4460 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 21:16:00.252578 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.252559    4460 manager.go:324] Recovery completed
Oct 31 21:16:00.252884 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.252859    4460 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 21:16:00.267641 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.267623    4460 cpu_manager.go:226] "Starting" policy="none"
Oct 31 21:16:00.267641 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.267638    4460 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 31 21:16:00.267803 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.267657    4460 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 31 21:16:00.267803 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.267792    4460 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 31 21:16:00.267896 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.267808    4460 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 31 21:16:00.267896 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.267830    4460 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 31 21:16:00.267896 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.267870    4460 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 31 21:16:00.267896 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.267879    4460 policy_none.go:50] "Start"
Oct 31 21:16:00.267896 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.267891    4460 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 31 21:16:00.268144 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.267911    4460 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 31 21:16:00.276665 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.276648    4460 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 31 21:16:00.276751 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.276668    4460 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 31 21:16:00.276751 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.276677    4460 policy_none.go:44] "Start"
Oct 31 21:16:00.281818 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.281786    4460 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 31 21:16:00.285710 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.285695    4460 manager.go:341] "Starting Device Plugin manager"
Oct 31 21:16:00.286207 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:16:00.286190    4460 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 31 21:16:00.286207 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.286207    4460 server.go:87] "Starting device plugin registration server"
Oct 31 21:16:00.286481 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.286467    4460 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 31 21:16:00.286544 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.286482    4460 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 31 21:16:00.286965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.286932    4460 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 31 21:16:00.287063 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.287033    4460 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 31 21:16:00.287125 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.287072    4460 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 31 21:16:00.287125 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.287079    4460 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 31 21:16:00.287225 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.287126    4460 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-31T21:16:00.287061969Z","UUID":"53ba7eb9-9b74-42f7-99e5-2d3572ce3e72","Handler":null,"Name":"","Endpoint":""}
Oct 31 21:16:00.288425 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.288406    4460 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 31 21:16:00.288509 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.288431    4460 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 31 21:16:00.288583 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:16:00.288543    4460 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 31 21:16:00.317314 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.317277    4460 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-157.ec2.internal"]
Oct 31 21:16:00.317779 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.317748    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal" event={"ID":"050c66d67c3ed1a5b46b1f7ad6044575","Type":"ContainerStarted","Data":"9a5e54d1cd30d9d0e53e74a4d14af2c07bc1481948b07d30ad77ac2591466b1c"}
Oct 31 21:16:00.317875 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.317788    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal" event={"ID":"050c66d67c3ed1a5b46b1f7ad6044575","Type":"ContainerDied","Data":"52dab86d69c25d3fe6273ec3333d4be9e53e40aedf149a68ff3ea8917e0fcc99"}
Oct 31 21:16:00.317875 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.317808    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal" event={"ID":"050c66d67c3ed1a5b46b1f7ad6044575","Type":"ContainerStarted","Data":"a1a687d9c746d237d1e4e3b121f6060ac051e41b732f5c7792cb1a49c504a9b3"}
Oct 31 21:16:00.317988 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.317956    4460 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal"
Oct 31 21:16:00.318038 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.318012    4460 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-157.ec2.internal"
Oct 31 21:16:00.347565 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.347546    4460 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 21:16:00.347631 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:16:00.347586    4460 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-136-157.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-157.ec2.internal"
Oct 31 21:16:00.348722 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.348699    4460 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 21:16:00.348798 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:16:00.348743    4460 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal"
Oct 31 21:16:00.386835 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.386820    4460 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 21:16:00.387977 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.387960    4460 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasSufficientMemory"
Oct 31 21:16:00.388060 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.387987    4460 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasNoDiskPressure"
Oct 31 21:16:00.388060 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.387998    4460 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasSufficientPID"
Oct 31 21:16:00.388060 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.388037    4460 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-136-157.ec2.internal"
Oct 31 21:16:00.427113 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.427058    4460 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-136-157.ec2.internal"
Oct 31 21:16:00.427195 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.427118    4460 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-136-157.ec2.internal"
Oct 31 21:16:00.438958 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.438916    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/050c66d67c3ed1a5b46b1f7ad6044575-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal\" (UID: \"050c66d67c3ed1a5b46b1f7ad6044575\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal"
Oct 31 21:16:00.439054 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.438962    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/050c66d67c3ed1a5b46b1f7ad6044575-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal\" (UID: \"050c66d67c3ed1a5b46b1f7ad6044575\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal"
Oct 31 21:16:00.439054 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.438990    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6034bd5835de1c87762018f93d657206-config\") pod \"kube-apiserver-proxy-ip-10-0-136-157.ec2.internal\" (UID: \"6034bd5835de1c87762018f93d657206\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-157.ec2.internal"
Oct 31 21:16:00.540093 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.540065    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/050c66d67c3ed1a5b46b1f7ad6044575-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal\" (UID: \"050c66d67c3ed1a5b46b1f7ad6044575\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal"
Oct 31 21:16:00.540093 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.540095    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/050c66d67c3ed1a5b46b1f7ad6044575-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal\" (UID: \"050c66d67c3ed1a5b46b1f7ad6044575\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal"
Oct 31 21:16:00.540240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.540110    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6034bd5835de1c87762018f93d657206-config\") pod \"kube-apiserver-proxy-ip-10-0-136-157.ec2.internal\" (UID: \"6034bd5835de1c87762018f93d657206\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-157.ec2.internal"
Oct 31 21:16:00.540240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.540159    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6034bd5835de1c87762018f93d657206-config\") pod \"kube-apiserver-proxy-ip-10-0-136-157.ec2.internal\" (UID: \"6034bd5835de1c87762018f93d657206\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-157.ec2.internal"
Oct 31 21:16:00.540240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.540184    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/050c66d67c3ed1a5b46b1f7ad6044575-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal\" (UID: \"050c66d67c3ed1a5b46b1f7ad6044575\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal"
Oct 31 21:16:00.540240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:00.540159    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/050c66d67c3ed1a5b46b1f7ad6044575-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal\" (UID: \"050c66d67c3ed1a5b46b1f7ad6044575\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal"
Oct 31 21:16:01.170503 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.170464    4460 apiserver.go:52] "Watching apiserver"
Oct 31 21:16:01.201863 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.201832    4460 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 31 21:16:01.202261 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.202242    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-9zw4h","openshift-multus/multus-44v6k","openshift-network-operator/iptables-alerter-4wdgb","kube-system/kube-apiserver-proxy-ip-10-0-136-157.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm","openshift-dns/node-resolver-wx67m","openshift-ingress-canary/ingress-canary-x8cxm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal","kube-system/konnectivity-agent-4qsqm","openshift-image-registry/node-ca-xhzwc","openshift-insights/insights-runtime-extractor-47zb8","openshift-network-diagnostics/network-check-target-bsvln","openshift-multus/multus-additional-cni-plugins-ftf4p","openshift-multus/network-metrics-daemon-bwhxr","openshift-ovn-kubernetes/ovnkube-node-4ssl9","openshift-cluster-node-tuning-operator/tuned-d5qv4","openshift-dns/dns-default-fl2tq"]
Oct 31 21:16:01.204879 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.204855    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhxr"
Oct 31 21:16:01.206680 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.206659    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:16:01.220543 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.220521    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 31 21:16:01.220680 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.220582    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 31 21:16:01.220680 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.220629    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 31 21:16:01.220680 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.220660    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 31 21:16:01.220893 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.220863    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 31 21:16:01.220959 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.220922    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 31 21:16:01.221075 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.221062    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 31 21:16:01.221139 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.221128    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 31 21:16:01.221192 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.221177    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 31 21:16:01.221259 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.221237    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 31 21:16:01.221259 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.221255    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6mmhq\""
Oct 31 21:16:01.221414 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.221316    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 31 21:16:01.221414 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.221392    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9gmld\""
Oct 31 21:16:01.221492 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.221435    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vddbx\""
Oct 31 21:16:01.221492 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.221471    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-rdjnn\""
Oct 31 21:16:01.236449 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.236428    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dwlts\""
Oct 31 21:16:01.236449 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.236449    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 31 21:16:01.236566 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.236466    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-j68t2\""
Oct 31 21:16:01.236714 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.236697    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cvv8l\""
Oct 31 21:16:01.236778 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.236726    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-sflcb\""
Oct 31 21:16:01.237198 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.236751    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 31 21:16:01.237285 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.237226    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-p85m4\""
Oct 31 21:16:01.237335 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.237323    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 31 21:16:01.237430 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.237408    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xsvd6\""
Oct 31 21:16:01.237574 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.237412    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-866d2\""
Oct 31 21:16:01.237631 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.237578    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-g9h45\""
Oct 31 21:16:01.237676 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.237423    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 31 21:16:01.237676 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.237649    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 31 21:16:01.237774 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.237749    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-xdqcm\""
Oct 31 21:16:01.237874 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.237775    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-nq8jq\""
Oct 31 21:16:01.237980 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.237963    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ddmwn\""
Oct 31 21:16:01.238060 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.237964    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 31 21:16:01.241001 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.240978    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:01.256795 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.256780    4460 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal"
Oct 31 21:16:01.281219 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.281193    4460 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 21:16:01.281294 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:16:01.281231    4460 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal"
Oct 31 21:16:01.282152 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.282139    4460 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 31 21:16:01.295648 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.295611    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 31 21:16:01.295648 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.295628    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 31 21:16:01.295648 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.295636    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 31 21:16:01.295811 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.295630    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 31 21:16:01.295811 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.295676    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 31 21:16:01.295811 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.295684    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 31 21:16:01.295811 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.295730    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 31 21:16:01.295811 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.295782    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 31 21:16:01.296046 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.295906    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 31 21:16:01.296046 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.295911    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 31 21:16:01.296046 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.295911    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 31 21:16:01.296046 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.295927    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 31 21:16:01.296046 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.295935    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 31 21:16:01.296046 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.295962    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 31 21:16:01.296046 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.295978    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 31 21:16:01.296370 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.296169    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 31 21:16:01.296370 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.296193    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 31 21:16:01.296370 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.296193    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 31 21:16:01.296370 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.296196    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 31 21:16:01.296370 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.296224    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 31 21:16:01.296370 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.296215    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 31 21:16:01.296370 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.296249    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 31 21:16:01.296370 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.296208    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 31 21:16:01.344081 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.344060    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0d542edf-e7a7-4898-bd81-17bd67e18003-iptables-alerter-script\") pod \"iptables-alerter-4wdgb\" (UID: \"0d542edf-e7a7-4898-bd81-17bd67e18003\") " pod="openshift-network-operator/iptables-alerter-4wdgb"
Oct 31 21:16:01.344180 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.344086    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-multus-cni-dir\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.344180 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.344102    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-run-systemd\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.344180 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.344115    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-run-ovn\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.344180 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.344130    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:16:01.344340 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.344198    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-device-dir\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:16:01.344340 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.344293    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-lib-modules\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.344340 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.344313    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e12af35-fa4d-4c09-b428-a559b5ccc296-tmp\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.344340 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.344331    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a6207663-1cfa-48d9-8382-12973bbf29fd-config-volume\") pod \"dns-default-fl2tq\" (UID: \"a6207663-1cfa-48d9-8382-12973bbf29fd\") " pod="openshift-dns/dns-default-fl2tq"
Oct 31 21:16:01.344465 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.344414    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e12af35-fa4d-4c09-b428-a559b5ccc296-tmp\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.344465 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.344438    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-run-k8s-cni-cncf-io\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.344552 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.344463    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0d542edf-e7a7-4898-bd81-17bd67e18003-iptables-alerter-script\") pod \"iptables-alerter-4wdgb\" (UID: \"0d542edf-e7a7-4898-bd81-17bd67e18003\") " pod="openshift-network-operator/iptables-alerter-4wdgb"
Oct 31 21:16:01.344552 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.344470    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-dbus\") pod \"global-pull-secret-syncer-9zw4h\" (UID: \"22daf2b3-91ca-462b-8714-0e0b0ebbcb88\") " pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:16:01.344552 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.344489    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-node-log\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.344552 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.344526    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-cni-netd\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.344745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.344564    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/26069598-5dc4-4c96-b55f-dd2f09884419-ovnkube-config\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.344745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.344583    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a6207663-1cfa-48d9-8382-12973bbf29fd-config-volume\") pod \"dns-default-fl2tq\" (UID: \"a6207663-1cfa-48d9-8382-12973bbf29fd\") " pod="openshift-dns/dns-default-fl2tq"
Oct 31 21:16:01.344745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.344592    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:16:01.344745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.344656    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-tuned\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.344745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.344691    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-var-lib-kubelet\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.344745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.344732    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-tuned\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.344745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.344734    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-multus-conf-dir\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.345016 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.344763    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-etc-kubernetes\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.345016 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.344800    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-kubernetes\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.345016 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.344800    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:16:01.345016 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.344815    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/26069598-5dc4-4c96-b55f-dd2f09884419-ovnkube-config\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.345016 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.344831    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-sysctl-conf\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.345016 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.344848    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-var-lib-kubelet\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.345016 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.344875    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-kubelet\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.345016 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.344908    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-run-netns\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.345016 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.344926    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-run-ovn-kubernetes\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.345016 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.344964    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1695694b-12a1-4a6c-b931-bf7dda9ec50e-data-volume\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:16:01.345016 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.344987    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-registration-dir\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:16:01.345401 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.345042    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1695694b-12a1-4a6c-b931-bf7dda9ec50e-data-volume\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:16:01.345401 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.345068    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjb7n\" (UniqueName: \"kubernetes.io/projected/3e254918-3cc1-4419-9820-37dff30f4445-kube-api-access-mjb7n\") pod \"network-metrics-daemon-bwhxr\" (UID: \"3e254918-3cc1-4419-9820-37dff30f4445\") " pod="openshift-multus/network-metrics-daemon-bwhxr"
Oct 31 21:16:01.345401 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.345098    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-sysconfig\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.345401 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.345184    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-run\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.345401 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.345225    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-var-lib-cni-bin\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.345401 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.345257    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-var-lib-cni-multus\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.345401 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.345290    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swpfh\" (UniqueName: \"kubernetes.io/projected/26069598-5dc4-4c96-b55f-dd2f09884419-kube-api-access-swpfh\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.345401 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.345311    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-cni-binary-copy\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:16:01.345401 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.345340    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-sys\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.345844 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.345441    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/91776cb8-cbd4-441a-8d4a-17e342717812-multus-daemon-config\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.345844 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.345466    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb6ph\" (UniqueName: \"kubernetes.io/projected/91776cb8-cbd4-441a-8d4a-17e342717812-kube-api-access-mb6ph\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.345844 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.345539    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-kubelet-config\") pod \"global-pull-secret-syncer-9zw4h\" (UID: \"22daf2b3-91ca-462b-8714-0e0b0ebbcb88\") " pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:16:01.345844 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.345592    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-cni-binary-copy\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:16:01.345844 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.345630    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6207663-1cfa-48d9-8382-12973bbf29fd-tmp-dir\") pod \"dns-default-fl2tq\" (UID: \"a6207663-1cfa-48d9-8382-12973bbf29fd\") " pod="openshift-dns/dns-default-fl2tq"
Oct 31 21:16:01.345844 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.345656    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk72s\" (UniqueName: \"kubernetes.io/projected/b65919f8-6cc9-4061-8a67-e1d5455c596d-kube-api-access-pk72s\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:16:01.345844 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.345681    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:16:01.345844 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.345683    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/91776cb8-cbd4-441a-8d4a-17e342717812-multus-daemon-config\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.345844 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.345706    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91776cb8-cbd4-441a-8d4a-17e342717812-cni-binary-copy\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.345844 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.345731    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-run-netns\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.345844 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.345733    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6207663-1cfa-48d9-8382-12973bbf29fd-tmp-dir\") pod \"dns-default-fl2tq\" (UID: \"a6207663-1cfa-48d9-8382-12973bbf29fd\") " pod="openshift-dns/dns-default-fl2tq"
Oct 31 21:16:01.345844 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.345757    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3d9a2e4e-3a81-4ae6-80ed-68542e996259-agent-certs\") pod \"konnectivity-agent-4qsqm\" (UID: \"3d9a2e4e-3a81-4ae6-80ed-68542e996259\") " pod="kube-system/konnectivity-agent-4qsqm"
Oct 31 21:16:01.345844 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.345783    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3d9a2e4e-3a81-4ae6-80ed-68542e996259-konnectivity-ca\") pod \"konnectivity-agent-4qsqm\" (UID: \"3d9a2e4e-3a81-4ae6-80ed-68542e996259\") " pod="kube-system/konnectivity-agent-4qsqm"
Oct 31 21:16:01.345844 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.345800    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/23da9927-8255-4380-a3f0-b7ea9298ad40-host\") pod \"node-ca-xhzwc\" (UID: \"23da9927-8255-4380-a3f0-b7ea9298ad40\") " pod="openshift-image-registry/node-ca-xhzwc"
Oct 31 21:16:01.345844 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.345820    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktdc9\" (UniqueName: \"kubernetes.io/projected/23da9927-8255-4380-a3f0-b7ea9298ad40-kube-api-access-ktdc9\") pod \"node-ca-xhzwc\" (UID: \"23da9927-8255-4380-a3f0-b7ea9298ad40\") " pod="openshift-image-registry/node-ca-xhzwc"
Oct 31 21:16:01.346482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.345864    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:16:01.346482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.345890    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.346482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.345920    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb42ddd8-d246-4bf3-ae35-27aa4584b163-tmp-dir\") pod \"node-resolver-wx67m\" (UID: \"bb42ddd8-d246-4bf3-ae35-27aa4584b163\") " pod="openshift-dns/node-resolver-wx67m"
Oct 31 21:16:01.346482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.345922    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91776cb8-cbd4-441a-8d4a-17e342717812-cni-binary-copy\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.346482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.345969    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-system-cni-dir\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.346482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.345989    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-cnibin\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.346482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.346005    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/23da9927-8255-4380-a3f0-b7ea9298ad40-serviceca\") pod \"node-ca-xhzwc\" (UID: \"23da9927-8255-4380-a3f0-b7ea9298ad40\") " pod="openshift-image-registry/node-ca-xhzwc"
Oct 31 21:16:01.346482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.346028    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-etc-openvswitch\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.346482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.346033    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3d9a2e4e-3a81-4ae6-80ed-68542e996259-konnectivity-ca\") pod \"konnectivity-agent-4qsqm\" (UID: \"3d9a2e4e-3a81-4ae6-80ed-68542e996259\") " pod="kube-system/konnectivity-agent-4qsqm"
Oct 31 21:16:01.346482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.346045    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb42ddd8-d246-4bf3-ae35-27aa4584b163-tmp-dir\") pod \"node-resolver-wx67m\" (UID: \"bb42ddd8-d246-4bf3-ae35-27aa4584b163\") " pod="openshift-dns/node-resolver-wx67m"
Oct 31 21:16:01.346482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.346080    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3d9a2e4e-3a81-4ae6-80ed-68542e996259-agent-certs\") pod \"konnectivity-agent-4qsqm\" (UID: \"3d9a2e4e-3a81-4ae6-80ed-68542e996259\") " pod="kube-system/konnectivity-agent-4qsqm"
Oct 31 21:16:01.346482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.346081    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-run-openvswitch\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.346482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.346108    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-socket-dir\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:16:01.346482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.346144    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c66dd\" (UniqueName: \"kubernetes.io/projected/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-kube-api-access-c66dd\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:16:01.346482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.346175    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-modprobe-d\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.346482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.346217    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-systemd\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.346482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.346249    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-original-pull-secret\") pod \"global-pull-secret-syncer-9zw4h\" (UID: \"22daf2b3-91ca-462b-8714-0e0b0ebbcb88\") " pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:16:01.346482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.346284    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-slash\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.346482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.346307    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-cni-bin\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.346482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.346335    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1695694b-12a1-4a6c-b931-bf7dda9ec50e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:16:01.346482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.346362    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-etc-selinux\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:16:01.346482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.346386    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-sys-fs\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:16:01.346482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.346409    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-sysctl-d\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.346482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.346431    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-cnibin\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:16:01.346482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.346464    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfq9g\" (UniqueName: \"kubernetes.io/projected/08d86dfb-548b-4a83-a1c2-d6ce8d4a0091-kube-api-access-pfq9g\") pod \"network-check-target-bsvln\" (UID: \"08d86dfb-548b-4a83-a1c2-d6ce8d4a0091\") " pod="openshift-network-diagnostics/network-check-target-bsvln"
Oct 31 21:16:01.347647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.346526    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-original-pull-secret\") pod \"global-pull-secret-syncer-9zw4h\" (UID: \"22daf2b3-91ca-462b-8714-0e0b0ebbcb88\") " pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:16:01.347647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.346534    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e254918-3cc1-4419-9820-37dff30f4445-metrics-certs\") pod \"network-metrics-daemon-bwhxr\" (UID: \"3e254918-3cc1-4419-9820-37dff30f4445\") " pod="openshift-multus/network-metrics-daemon-bwhxr"
Oct 31 21:16:01.347647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.346568    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbz7b\" (UniqueName: \"kubernetes.io/projected/5e12af35-fa4d-4c09-b428-a559b5ccc296-kube-api-access-cbz7b\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.347647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.346587    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97qrk\" (UniqueName: \"kubernetes.io/projected/6d63e78f-f474-4966-adb5-401be54f4ab4-kube-api-access-97qrk\") pod \"ingress-canary-x8cxm\" (UID: \"6d63e78f-f474-4966-adb5-401be54f4ab4\") " pod="openshift-ingress-canary/ingress-canary-x8cxm"
Oct 31 21:16:01.347647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.346611    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-var-lib-openvswitch\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.347647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.346639    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/23da9927-8255-4380-a3f0-b7ea9298ad40-serviceca\") pod \"node-ca-xhzwc\" (UID: \"23da9927-8255-4380-a3f0-b7ea9298ad40\") " pod="openshift-image-registry/node-ca-xhzwc"
Oct 31 21:16:01.347647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.346640    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/26069598-5dc4-4c96-b55f-dd2f09884419-env-overrides\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.347647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.346677    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/26069598-5dc4-4c96-b55f-dd2f09884419-ovnkube-script-lib\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.347647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.346694    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6207663-1cfa-48d9-8382-12973bbf29fd-metrics-tls\") pod \"dns-default-fl2tq\" (UID: \"a6207663-1cfa-48d9-8382-12973bbf29fd\") " pod="openshift-dns/dns-default-fl2tq"
Oct 31 21:16:01.347647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.346723    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-system-cni-dir\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:16:01.347647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.346829    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/26069598-5dc4-4c96-b55f-dd2f09884419-env-overrides\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.347647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.347032    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a6207663-1cfa-48d9-8382-12973bbf29fd-metrics-tls\") pod \"dns-default-fl2tq\" (UID: \"a6207663-1cfa-48d9-8382-12973bbf29fd\") " pod="openshift-dns/dns-default-fl2tq"
Oct 31 21:16:01.347647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.347162    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/26069598-5dc4-4c96-b55f-dd2f09884419-ovnkube-script-lib\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.347647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.347207    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnrb2\" (UniqueName: \"kubernetes.io/projected/0d542edf-e7a7-4898-bd81-17bd67e18003-kube-api-access-lnrb2\") pod \"iptables-alerter-4wdgb\" (UID: \"0d542edf-e7a7-4898-bd81-17bd67e18003\") " pod="openshift-network-operator/iptables-alerter-4wdgb"
Oct 31 21:16:01.347647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.347236    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bb42ddd8-d246-4bf3-ae35-27aa4584b163-hosts-file\") pod \"node-resolver-wx67m\" (UID: \"bb42ddd8-d246-4bf3-ae35-27aa4584b163\") " pod="openshift-dns/node-resolver-wx67m"
Oct 31 21:16:01.347647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.347352    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rh6z\" (UniqueName: \"kubernetes.io/projected/bb42ddd8-d246-4bf3-ae35-27aa4584b163-kube-api-access-2rh6z\") pod \"node-resolver-wx67m\" (UID: \"bb42ddd8-d246-4bf3-ae35-27aa4584b163\") " pod="openshift-dns/node-resolver-wx67m"
Oct 31 21:16:01.347647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.347384    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-hostroot\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.347647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.347403    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-run-multus-certs\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.347647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.347443    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-systemd-units\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.347647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.347494    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1695694b-12a1-4a6c-b931-bf7dda9ec50e-crio-socket\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:16:01.347647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.347518    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pknd\" (UniqueName: \"kubernetes.io/projected/1695694b-12a1-4a6c-b931-bf7dda9ec50e-kube-api-access-9pknd\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:16:01.347647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.347534    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-host\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.347647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.347549    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-multus-socket-dir-parent\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.347647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.347585    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/26069598-5dc4-4c96-b55f-dd2f09884419-ovn-node-metrics-cert\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.347647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.347614    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0d542edf-e7a7-4898-bd81-17bd67e18003-host-slash\") pod \"iptables-alerter-4wdgb\" (UID: \"0d542edf-e7a7-4898-bd81-17bd67e18003\") " pod="openshift-network-operator/iptables-alerter-4wdgb"
Oct 31 21:16:01.347647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.347629    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-log-socket\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.348481 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.347705    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1695694b-12a1-4a6c-b931-bf7dda9ec50e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:16:01.348481 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.347750    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnxs9\" (UniqueName: \"kubernetes.io/projected/a6207663-1cfa-48d9-8382-12973bbf29fd-kube-api-access-mnxs9\") pod \"dns-default-fl2tq\" (UID: \"a6207663-1cfa-48d9-8382-12973bbf29fd\") " pod="openshift-dns/dns-default-fl2tq"
Oct 31 21:16:01.348481 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.347846    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d63e78f-f474-4966-adb5-401be54f4ab4-cert\") pod \"ingress-canary-x8cxm\" (UID: \"6d63e78f-f474-4966-adb5-401be54f4ab4\") " pod="openshift-ingress-canary/ingress-canary-x8cxm"
Oct 31 21:16:01.348481 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.347879    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:16:01.348481 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.347933    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1695694b-12a1-4a6c-b931-bf7dda9ec50e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:16:01.348481 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.347931    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/26069598-5dc4-4c96-b55f-dd2f09884419-ovn-node-metrics-cert\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.348481 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.348040    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d63e78f-f474-4966-adb5-401be54f4ab4-cert\") pod \"ingress-canary-x8cxm\" (UID: \"6d63e78f-f474-4966-adb5-401be54f4ab4\") " pod="openshift-ingress-canary/ingress-canary-x8cxm"
Oct 31 21:16:01.361340 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.361316    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:01.372304 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.372279    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pknd\" (UniqueName: \"kubernetes.io/projected/1695694b-12a1-4a6c-b931-bf7dda9ec50e-kube-api-access-9pknd\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:16:01.372396 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.372375    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rh6z\" (UniqueName: \"kubernetes.io/projected/bb42ddd8-d246-4bf3-ae35-27aa4584b163-kube-api-access-2rh6z\") pod \"node-resolver-wx67m\" (UID: \"bb42ddd8-d246-4bf3-ae35-27aa4584b163\") " pod="openshift-dns/node-resolver-wx67m"
Oct 31 21:16:01.372633 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.372591    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktdc9\" (UniqueName: \"kubernetes.io/projected/23da9927-8255-4380-a3f0-b7ea9298ad40-kube-api-access-ktdc9\") pod \"node-ca-xhzwc\" (UID: \"23da9927-8255-4380-a3f0-b7ea9298ad40\") " pod="openshift-image-registry/node-ca-xhzwc"
Oct 31 21:16:01.373218 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.372841    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfq9g\" (UniqueName: \"kubernetes.io/projected/08d86dfb-548b-4a83-a1c2-d6ce8d4a0091-kube-api-access-pfq9g\") pod \"network-check-target-bsvln\" (UID: \"08d86dfb-548b-4a83-a1c2-d6ce8d4a0091\") " pod="openshift-network-diagnostics/network-check-target-bsvln"
Oct 31 21:16:01.373218 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.373055    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbz7b\" (UniqueName: \"kubernetes.io/projected/5e12af35-fa4d-4c09-b428-a559b5ccc296-kube-api-access-cbz7b\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.373218 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.373084    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb6ph\" (UniqueName: \"kubernetes.io/projected/91776cb8-cbd4-441a-8d4a-17e342717812-kube-api-access-mb6ph\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.373218 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.373178    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnxs9\" (UniqueName: \"kubernetes.io/projected/a6207663-1cfa-48d9-8382-12973bbf29fd-kube-api-access-mnxs9\") pod \"dns-default-fl2tq\" (UID: \"a6207663-1cfa-48d9-8382-12973bbf29fd\") " pod="openshift-dns/dns-default-fl2tq"
Oct 31 21:16:01.373218 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.373188    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c66dd\" (UniqueName: \"kubernetes.io/projected/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-kube-api-access-c66dd\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:16:01.373422 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.373172    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk72s\" (UniqueName: \"kubernetes.io/projected/b65919f8-6cc9-4061-8a67-e1d5455c596d-kube-api-access-pk72s\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:16:01.373422 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.373230    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-swpfh\" (UniqueName: \"kubernetes.io/projected/26069598-5dc4-4c96-b55f-dd2f09884419-kube-api-access-swpfh\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.373422 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.373256    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjb7n\" (UniqueName: \"kubernetes.io/projected/3e254918-3cc1-4419-9820-37dff30f4445-kube-api-access-mjb7n\") pod \"network-metrics-daemon-bwhxr\" (UID: \"3e254918-3cc1-4419-9820-37dff30f4445\") " pod="openshift-multus/network-metrics-daemon-bwhxr"
Oct 31 21:16:01.373422 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.373302    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnrb2\" (UniqueName: \"kubernetes.io/projected/0d542edf-e7a7-4898-bd81-17bd67e18003-kube-api-access-lnrb2\") pod \"iptables-alerter-4wdgb\" (UID: \"0d542edf-e7a7-4898-bd81-17bd67e18003\") " pod="openshift-network-operator/iptables-alerter-4wdgb"
Oct 31 21:16:01.373589 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.373573    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97qrk\" (UniqueName: \"kubernetes.io/projected/6d63e78f-f474-4966-adb5-401be54f4ab4-kube-api-access-97qrk\") pod \"ingress-canary-x8cxm\" (UID: \"6d63e78f-f474-4966-adb5-401be54f4ab4\") " pod="openshift-ingress-canary/ingress-canary-x8cxm"
Oct 31 21:16:01.448806 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.448742    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-socket-dir\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:16:01.448806 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.448774    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-modprobe-d\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.448806 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.448796    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-systemd\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.449039 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.448818    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-slash\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.449039 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.448841    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-cni-bin\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.449039 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.448863    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1695694b-12a1-4a6c-b931-bf7dda9ec50e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:16:01.449039 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.448867    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-socket-dir\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:16:01.449039 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.448890    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-etc-selinux\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:16:01.449039 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.448891    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-systemd\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.449039 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.448891    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-slash\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.449039 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.448910    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-cni-bin\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.449039 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.448923    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-modprobe-d\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.449039 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.448912    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-sys-fs\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:16:01.449039 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.448979    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-etc-selinux\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:16:01.449039 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.448985    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-sysctl-d\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.449039 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.448990    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-sys-fs\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:16:01.449039 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449026    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-cnibin\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449056    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-sysctl-d\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449054    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e254918-3cc1-4419-9820-37dff30f4445-metrics-certs\") pod \"network-metrics-daemon-bwhxr\" (UID: \"3e254918-3cc1-4419-9820-37dff30f4445\") " pod="openshift-multus/network-metrics-daemon-bwhxr"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449087    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-var-lib-openvswitch\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449089    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-cnibin\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449106    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-system-cni-dir\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449122    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bb42ddd8-d246-4bf3-ae35-27aa4584b163-hosts-file\") pod \"node-resolver-wx67m\" (UID: \"bb42ddd8-d246-4bf3-ae35-27aa4584b163\") " pod="openshift-dns/node-resolver-wx67m"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449132    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-var-lib-openvswitch\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449137    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-hostroot\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449155    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-hostroot\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449169    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-run-multus-certs\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449192    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-systemd-units\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449193    4460 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449204    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bb42ddd8-d246-4bf3-ae35-27aa4584b163-hosts-file\") pod \"node-resolver-wx67m\" (UID: \"bb42ddd8-d246-4bf3-ae35-27aa4584b163\") " pod="openshift-dns/node-resolver-wx67m"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449213    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1695694b-12a1-4a6c-b931-bf7dda9ec50e-crio-socket\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449205    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-system-cni-dir\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449226    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-systemd-units\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449237    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-host\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449198    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-run-multus-certs\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449264    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-multus-socket-dir-parent\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449268    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1695694b-12a1-4a6c-b931-bf7dda9ec50e-crio-socket\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449286    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-host\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449289    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0d542edf-e7a7-4898-bd81-17bd67e18003-host-slash\") pod \"iptables-alerter-4wdgb\" (UID: \"0d542edf-e7a7-4898-bd81-17bd67e18003\") " pod="openshift-network-operator/iptables-alerter-4wdgb"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449319    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-log-socket\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449322    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0d542edf-e7a7-4898-bd81-17bd67e18003-host-slash\") pod \"iptables-alerter-4wdgb\" (UID: \"0d542edf-e7a7-4898-bd81-17bd67e18003\") " pod="openshift-network-operator/iptables-alerter-4wdgb"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449320    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-multus-socket-dir-parent\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449335    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449369    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-multus-cni-dir\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449391    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-run-systemd\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449372    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/94b754a2-a8f3-494c-89bb-e903f9c2cdb0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ftf4p\" (UID: \"94b754a2-a8f3-494c-89bb-e903f9c2cdb0\") " pod="openshift-multus/multus-additional-cni-plugins-ftf4p"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449413    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-run-ovn\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449429    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-multus-cni-dir\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449437    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-run-ovn\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449403    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-log-socket\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449437    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-run-systemd\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449477    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449507    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-device-dir\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449531    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-lib-modules\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449555    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-run-k8s-cni-cncf-io\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449546    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449568    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-device-dir\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449578    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-dbus\") pod \"global-pull-secret-syncer-9zw4h\" (UID: \"22daf2b3-91ca-462b-8714-0e0b0ebbcb88\") " pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:16:01.449584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449604    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-node-log\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449618    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-run-k8s-cni-cncf-io\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449625    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-cni-netd\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449660    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-var-lib-kubelet\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449664    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-lib-modules\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449680    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-multus-conf-dir\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449690    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-cni-netd\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449696    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-etc-kubernetes\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449718    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-etc-kubernetes\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449665    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-node-log\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449719    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-dbus\") pod \"global-pull-secret-syncer-9zw4h\" (UID: \"22daf2b3-91ca-462b-8714-0e0b0ebbcb88\") " pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449728    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-kubernetes\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449759    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-sysctl-conf\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449764    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-var-lib-kubelet\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449797    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-kubernetes\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449850    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-multus-conf-dir\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449874    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-var-lib-kubelet\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449894    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-sysctl-conf\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449902    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-kubelet\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449972    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-run-netns\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449987    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-var-lib-kubelet\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.449997    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-run-ovn-kubernetes\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.450013    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-kubelet\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.450037    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-run-netns\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.450040    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-registration-dir\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.450074    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-sysconfig\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.450088    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-run-ovn-kubernetes\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.450096    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-run\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.450097    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b65919f8-6cc9-4061-8a67-e1d5455c596d-registration-dir\") pod \"aws-ebs-csi-driver-node-vpqsm\" (UID: \"b65919f8-6cc9-4061-8a67-e1d5455c596d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vpqsm"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.450153    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-run\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.450197    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-etc-sysconfig\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.450225    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-var-lib-cni-bin\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.450249    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-var-lib-cni-multus\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.450275    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-sys\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.450299    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-kubelet-config\") pod \"global-pull-secret-syncer-9zw4h\" (UID: \"22daf2b3-91ca-462b-8714-0e0b0ebbcb88\") " pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.450323    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-var-lib-cni-multus\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.450325    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-run-netns\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.450347    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-var-lib-cni-bin\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.450361    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-host-run-netns\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.450366    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/23da9927-8255-4380-a3f0-b7ea9298ad40-host\") pod \"node-ca-xhzwc\" (UID: \"23da9927-8255-4380-a3f0-b7ea9298ad40\") " pod="openshift-image-registry/node-ca-xhzwc"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.450326    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5e12af35-fa4d-4c09-b428-a559b5ccc296-sys\") pod \"tuned-d5qv4\" (UID: \"5e12af35-fa4d-4c09-b428-a559b5ccc296\") " pod="openshift-cluster-node-tuning-operator/tuned-d5qv4"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.450397    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.450370    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/22daf2b3-91ca-462b-8714-0e0b0ebbcb88-kubelet-config\") pod \"global-pull-secret-syncer-9zw4h\" (UID: \"22daf2b3-91ca-462b-8714-0e0b0ebbcb88\") " pod="kube-system/global-pull-secret-syncer-9zw4h"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.450422    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-system-cni-dir\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.450422    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/23da9927-8255-4380-a3f0-b7ea9298ad40-host\") pod \"node-ca-xhzwc\" (UID: \"23da9927-8255-4380-a3f0-b7ea9298ad40\") " pod="openshift-image-registry/node-ca-xhzwc"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.450451    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.450451    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-cnibin\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.450475    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-etc-openvswitch\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.450493    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-cnibin\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.450496    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-run-openvswitch\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.450505    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91776cb8-cbd4-441a-8d4a-17e342717812-system-cni-dir\") pod \"multus-44v6k\" (UID: \"91776cb8-cbd4-441a-8d4a-17e342717812\") " pod="openshift-multus/multus-44v6k"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.450539    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-etc-openvswitch\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.451053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.450543    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/26069598-5dc4-4c96-b55f-dd2f09884419-run-openvswitch\") pod \"ovnkube-node-4ssl9\" (UID: \"26069598-5dc4-4c96-b55f-dd2f09884419\") " pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:01.452451 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.451415    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1695694b-12a1-4a6c-b931-bf7dda9ec50e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-47zb8\" (UID: \"1695694b-12a1-4a6c-b931-bf7dda9ec50e\") " pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:16:01.452451 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.451459    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e254918-3cc1-4419-9820-37dff30f4445-metrics-certs\") pod \"network-metrics-daemon-bwhxr\" (UID: \"3e254918-3cc1-4419-9820-37dff30f4445\") " pod="openshift-multus/network-metrics-daemon-bwhxr"
Oct 31 21:16:01.505634 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.505604    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bwhxr"
Oct 31 21:16:01.507914 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.507895    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-47zb8"
Oct 31 21:16:01.511400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.511372    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:01.663858 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.663825    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:01.775748 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.775712    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:01.818525 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.818495    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bwhxr"]
Oct 31 21:16:01.821495 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:01.821468    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e254918_3cc1_4419_9820_37dff30f4445.slice/crio-84369331fbac732502f8ca558024395b30596949f499dcf7a99422e839d3e10d WatchSource:0}: Error finding container 84369331fbac732502f8ca558024395b30596949f499dcf7a99422e839d3e10d: Status 404 returned error can't find the container with id 84369331fbac732502f8ca558024395b30596949f499dcf7a99422e839d3e10d
Oct 31 21:16:01.823646 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.823631    4460 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 21:16:01.844532 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.844511    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-47zb8"]
Oct 31 21:16:01.846992 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:01.846972    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1695694b_12a1_4a6c_b931_bf7dda9ec50e.slice/crio-cd29cf4abdb27fd3390507716e8c5d4391fdf28311550f2de12b3e905868f6e5 WatchSource:0}: Error finding container cd29cf4abdb27fd3390507716e8c5d4391fdf28311550f2de12b3e905868f6e5: Status 404 returned error can't find the container with id cd29cf4abdb27fd3390507716e8c5d4391fdf28311550f2de12b3e905868f6e5
Oct 31 21:16:01.887866 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:01.887840    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:02.161825 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:02.161745    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:02.262367 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:02.262334    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-47zb8" event={"ID":"1695694b-12a1-4a6c-b931-bf7dda9ec50e","Type":"ContainerStarted","Data":"70539f0f4d096c157d2680bb12592d6affdffa3e206e7d8e45d57dd7505a1786"}
Oct 31 21:16:02.262812 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:02.262397    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-47zb8" event={"ID":"1695694b-12a1-4a6c-b931-bf7dda9ec50e","Type":"ContainerStarted","Data":"cd29cf4abdb27fd3390507716e8c5d4391fdf28311550f2de12b3e905868f6e5"}
Oct 31 21:16:02.263890 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:02.263863    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bwhxr" event={"ID":"3e254918-3cc1-4419-9820-37dff30f4445","Type":"ContainerStarted","Data":"84369331fbac732502f8ca558024395b30596949f499dcf7a99422e839d3e10d"}
Oct 31 21:16:02.817909 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:02.817883    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:03.147335 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:03.147257    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-fl2tq"
Oct 31 21:16:03.271134 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:03.271101    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bwhxr" event={"ID":"3e254918-3cc1-4419-9820-37dff30f4445","Type":"ContainerStarted","Data":"3b2869ba7d15a0868cb49ab86a20cb9cc5c7768178f4378369abcec7c79e77d7"}
Oct 31 21:16:03.271649 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:03.271211    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bwhxr" event={"ID":"3e254918-3cc1-4419-9820-37dff30f4445","Type":"ContainerStarted","Data":"48d085e0a0a3f69962706debd1a9537b595b09084828b1d3982a44cd72e5c5e2"}
Oct 31 21:16:03.272869 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:03.272839    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-47zb8" event={"ID":"1695694b-12a1-4a6c-b931-bf7dda9ec50e","Type":"ContainerStarted","Data":"5218764f0004f352701d80412ff491b42937c5e1a7d62218ee806a5e65e1955f"}
Oct 31 21:16:04.787105 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:04.787075    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-bsvln"
Oct 31 21:16:05.281721 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:05.281689    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-47zb8" event={"ID":"1695694b-12a1-4a6c-b931-bf7dda9ec50e","Type":"ContainerStarted","Data":"b6ecc33a1a35cf276c2400d593ca39b08f1f62241b9ec924116e932525cae7c2"}
Oct 31 21:16:09.149345 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:09.149317    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-fl2tq"
Oct 31 21:16:09.850553 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:09.850516    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:10.047272 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:10.047230    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:10.163021 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:10.162925    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:10.274169 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:10.274133    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:10.497346 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:10.497261    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:10.497567 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:10.497507    4460 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 31 21:16:11.186013 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:11.185979    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:11.299717 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:11.299684    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:11.584541 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:11.584510    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:11.711407 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:11.711372    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:13.769092 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:13.769054    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:13.879803 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:13.879763    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:13.893149 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:13.893121    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-xnn8z"]
Oct 31 21:16:13.895022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:13.895007    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-xnn8z"
Oct 31 21:16:13.912989 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:13.912965    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 31 21:16:13.915062 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:13.915043    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 31 21:16:13.926862 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:13.926841    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-cn9dp\""
Oct 31 21:16:13.929365 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:13.929337    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 31 21:16:13.929365 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:13.929340    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 31 21:16:13.929918 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:13.929897    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-xnn8z"]
Oct 31 21:16:13.940375 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:13.940361    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 31 21:16:14.024470 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:14.024380    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d0b7876-45a7-4c96-be68-26417878ea42-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-xnn8z\" (UID: \"8d0b7876-45a7-4c96-be68-26417878ea42\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-xnn8z"
Oct 31 21:16:14.024470 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:14.024419    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8d0b7876-45a7-4c96-be68-26417878ea42-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-xnn8z\" (UID: \"8d0b7876-45a7-4c96-be68-26417878ea42\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-xnn8z"
Oct 31 21:16:14.024470 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:14.024446    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8d0b7876-45a7-4c96-be68-26417878ea42-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-xnn8z\" (UID: \"8d0b7876-45a7-4c96-be68-26417878ea42\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-xnn8z"
Oct 31 21:16:14.024675 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:14.024502    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq8tn\" (UniqueName: \"kubernetes.io/projected/8d0b7876-45a7-4c96-be68-26417878ea42-kube-api-access-cq8tn\") pod \"prometheus-operator-9cd89b74c-xnn8z\" (UID: \"8d0b7876-45a7-4c96-be68-26417878ea42\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-xnn8z"
Oct 31 21:16:14.125002 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:14.124967    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d0b7876-45a7-4c96-be68-26417878ea42-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-xnn8z\" (UID: \"8d0b7876-45a7-4c96-be68-26417878ea42\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-xnn8z"
Oct 31 21:16:14.125192 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:14.125011    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8d0b7876-45a7-4c96-be68-26417878ea42-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-xnn8z\" (UID: \"8d0b7876-45a7-4c96-be68-26417878ea42\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-xnn8z"
Oct 31 21:16:14.125192 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:14.125145    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8d0b7876-45a7-4c96-be68-26417878ea42-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-xnn8z\" (UID: \"8d0b7876-45a7-4c96-be68-26417878ea42\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-xnn8z"
Oct 31 21:16:14.125192 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:14.125174    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq8tn\" (UniqueName: \"kubernetes.io/projected/8d0b7876-45a7-4c96-be68-26417878ea42-kube-api-access-cq8tn\") pod \"prometheus-operator-9cd89b74c-xnn8z\" (UID: \"8d0b7876-45a7-4c96-be68-26417878ea42\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-xnn8z"
Oct 31 21:16:14.125586 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:14.125552    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:14.125832 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:14.125804    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8d0b7876-45a7-4c96-be68-26417878ea42-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-xnn8z\" (UID: \"8d0b7876-45a7-4c96-be68-26417878ea42\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-xnn8z"
Oct 31 21:16:14.127590 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:14.127567    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/8d0b7876-45a7-4c96-be68-26417878ea42-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-xnn8z\" (UID: \"8d0b7876-45a7-4c96-be68-26417878ea42\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-xnn8z"
Oct 31 21:16:14.127671 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:14.127592    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8d0b7876-45a7-4c96-be68-26417878ea42-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-xnn8z\" (UID: \"8d0b7876-45a7-4c96-be68-26417878ea42\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-xnn8z"
Oct 31 21:16:14.152455 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:14.152426    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq8tn\" (UniqueName: \"kubernetes.io/projected/8d0b7876-45a7-4c96-be68-26417878ea42-kube-api-access-cq8tn\") pod \"prometheus-operator-9cd89b74c-xnn8z\" (UID: \"8d0b7876-45a7-4c96-be68-26417878ea42\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-xnn8z"
Oct 31 21:16:14.205474 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:14.205444    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-xnn8z"
Oct 31 21:16:14.249104 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:14.249070    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:14.497463 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:14.497392    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-xnn8z"]
Oct 31 21:16:14.500325 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:14.500302    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d0b7876_45a7_4c96_be68_26417878ea42.slice/crio-7816b11508d567e3c9322b337afec5339cf50244c86152c296cbdbed8a22e2e0 WatchSource:0}: Error finding container 7816b11508d567e3c9322b337afec5339cf50244c86152c296cbdbed8a22e2e0: Status 404 returned error can't find the container with id 7816b11508d567e3c9322b337afec5339cf50244c86152c296cbdbed8a22e2e0
Oct 31 21:16:15.309983 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:15.309931    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-xnn8z" event={"ID":"8d0b7876-45a7-4c96-be68-26417878ea42","Type":"ContainerStarted","Data":"7816b11508d567e3c9322b337afec5339cf50244c86152c296cbdbed8a22e2e0"}
Oct 31 21:16:16.316050 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:16.316011    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-xnn8z" event={"ID":"8d0b7876-45a7-4c96-be68-26417878ea42","Type":"ContainerStarted","Data":"98cc2d8eaaba7ceeccc5cbc7da8fffe17f3b264a76a01c7995ae0b8882f5a9e3"}
Oct 31 21:16:16.316538 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:16.316060    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-xnn8z" event={"ID":"8d0b7876-45a7-4c96-be68-26417878ea42","Type":"ContainerStarted","Data":"f7e9cd8653287472586086dd38f1ae6148efdba24eb467a07acdf161a3cdd3af"}
Oct 31 21:16:16.411044 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:16.411007    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:17.229089 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:17.229052    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:17.345442 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:17.345404    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:18.253352 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.253315    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:18.424507 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.424466    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:18.531108 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.531043    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-xnn8z" podStartSLOduration=4.35952334 podStartE2EDuration="5.53102847s" totalImagesPullingTime="1.17150513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:16:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:16:16.361127619 +0000 UTC m=+16.277437048" watchObservedRunningTime="2026-10-31 21:16:18.53102847 +0000 UTC m=+18.447337900"
Oct 31 21:16:18.531378 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.531363    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-kl9j9"]
Oct 31 21:16:18.533361 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.533345    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-kl9j9"
Oct 31 21:16:18.546246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.546217    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:18.556344 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.556316    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-kl9j9"]
Oct 31 21:16:18.569138 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.569108    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 31 21:16:18.569138 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.569109    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 31 21:16:18.569359 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.569229    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-snqk2\""
Oct 31 21:16:18.570051 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.570030    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-r7cc2"]
Oct 31 21:16:18.572323 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.572303    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-r7cc2"
Oct 31 21:16:18.595225 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.595198    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 31 21:16:18.596260 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.596235    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-szhn4"]
Oct 31 21:16:18.598490 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.598469    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-r7cc2"]
Oct 31 21:16:18.598607 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.598588    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-szhn4"
Oct 31 21:16:18.600765 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.600741    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 31 21:16:18.608352 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.608334    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 31 21:16:18.608437 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.608339    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-sjk55\""
Oct 31 21:16:18.636777 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.636747    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 31 21:16:18.636777 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.636747    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xn5d6\""
Oct 31 21:16:18.636984 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.636853    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 31 21:16:18.639685 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.639660    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 31 21:16:18.659275 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.659239    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d98a0acd-99d1-498f-a483-12ea20459971-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-kl9j9\" (UID: \"d98a0acd-99d1-498f-a483-12ea20459971\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-kl9j9"
Oct 31 21:16:18.659450 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.659282    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d98a0acd-99d1-498f-a483-12ea20459971-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-kl9j9\" (UID: \"d98a0acd-99d1-498f-a483-12ea20459971\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-kl9j9"
Oct 31 21:16:18.659450 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.659309    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqmnk\" (UniqueName: \"kubernetes.io/projected/d98a0acd-99d1-498f-a483-12ea20459971-kube-api-access-xqmnk\") pod \"openshift-state-metrics-7bd7489957-kl9j9\" (UID: \"d98a0acd-99d1-498f-a483-12ea20459971\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-kl9j9"
Oct 31 21:16:18.659450 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.659415    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d98a0acd-99d1-498f-a483-12ea20459971-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-kl9j9\" (UID: \"d98a0acd-99d1-498f-a483-12ea20459971\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-kl9j9"
Oct 31 21:16:18.663528 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.663501    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:18.760112 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.760080    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d98a0acd-99d1-498f-a483-12ea20459971-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-kl9j9\" (UID: \"d98a0acd-99d1-498f-a483-12ea20459971\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-kl9j9"
Oct 31 21:16:18.760112 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.760118    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e31eea5-fbe9-47c4-98fa-f2e9af6239e4-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-r7cc2\" (UID: \"0e31eea5-fbe9-47c4-98fa-f2e9af6239e4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-r7cc2"
Oct 31 21:16:18.760315 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.760151    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d7030dce-004e-405d-8789-014516345b75-node-exporter-textfile\") pod \"node-exporter-szhn4\" (UID: \"d7030dce-004e-405d-8789-014516345b75\") " pod="openshift-monitoring/node-exporter-szhn4"
Oct 31 21:16:18.760315 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.760177    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0e31eea5-fbe9-47c4-98fa-f2e9af6239e4-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-r7cc2\" (UID: \"0e31eea5-fbe9-47c4-98fa-f2e9af6239e4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-r7cc2"
Oct 31 21:16:18.760315 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.760244    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d7030dce-004e-405d-8789-014516345b75-sys\") pod \"node-exporter-szhn4\" (UID: \"d7030dce-004e-405d-8789-014516345b75\") " pod="openshift-monitoring/node-exporter-szhn4"
Oct 31 21:16:18.760406 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.760370    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d7030dce-004e-405d-8789-014516345b75-root\") pod \"node-exporter-szhn4\" (UID: \"d7030dce-004e-405d-8789-014516345b75\") " pod="openshift-monitoring/node-exporter-szhn4"
Oct 31 21:16:18.760442 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.760403    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d98a0acd-99d1-498f-a483-12ea20459971-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-kl9j9\" (UID: \"d98a0acd-99d1-498f-a483-12ea20459971\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-kl9j9"
Oct 31 21:16:18.760442 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.760428    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7030dce-004e-405d-8789-014516345b75-metrics-client-ca\") pod \"node-exporter-szhn4\" (UID: \"d7030dce-004e-405d-8789-014516345b75\") " pod="openshift-monitoring/node-exporter-szhn4"
Oct 31 21:16:18.760501 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.760470    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nld6t\" (UniqueName: \"kubernetes.io/projected/d7030dce-004e-405d-8789-014516345b75-kube-api-access-nld6t\") pod \"node-exporter-szhn4\" (UID: \"d7030dce-004e-405d-8789-014516345b75\") " pod="openshift-monitoring/node-exporter-szhn4"
Oct 31 21:16:18.760576 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.760559    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlcp2\" (UniqueName: \"kubernetes.io/projected/0e31eea5-fbe9-47c4-98fa-f2e9af6239e4-kube-api-access-qlcp2\") pod \"kube-state-metrics-5dc8d7d89b-r7cc2\" (UID: \"0e31eea5-fbe9-47c4-98fa-f2e9af6239e4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-r7cc2"
Oct 31 21:16:18.760621 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.760595    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e31eea5-fbe9-47c4-98fa-f2e9af6239e4-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-r7cc2\" (UID: \"0e31eea5-fbe9-47c4-98fa-f2e9af6239e4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-r7cc2"
Oct 31 21:16:18.760621 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.760615    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0e31eea5-fbe9-47c4-98fa-f2e9af6239e4-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-r7cc2\" (UID: \"0e31eea5-fbe9-47c4-98fa-f2e9af6239e4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-r7cc2"
Oct 31 21:16:18.760719 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.760669    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d98a0acd-99d1-498f-a483-12ea20459971-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-kl9j9\" (UID: \"d98a0acd-99d1-498f-a483-12ea20459971\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-kl9j9"
Oct 31 21:16:18.760768 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.760750    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e31eea5-fbe9-47c4-98fa-f2e9af6239e4-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-r7cc2\" (UID: \"0e31eea5-fbe9-47c4-98fa-f2e9af6239e4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-r7cc2"
Oct 31 21:16:18.760819 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.760787    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d7030dce-004e-405d-8789-014516345b75-node-exporter-tls\") pod \"node-exporter-szhn4\" (UID: \"d7030dce-004e-405d-8789-014516345b75\") " pod="openshift-monitoring/node-exporter-szhn4"
Oct 31 21:16:18.760876 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.760816    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d7030dce-004e-405d-8789-014516345b75-node-exporter-accelerators-collector-config\") pod \"node-exporter-szhn4\" (UID: \"d7030dce-004e-405d-8789-014516345b75\") " pod="openshift-monitoring/node-exporter-szhn4"
Oct 31 21:16:18.760876 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.760834    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d7030dce-004e-405d-8789-014516345b75-node-exporter-wtmp\") pod \"node-exporter-szhn4\" (UID: \"d7030dce-004e-405d-8789-014516345b75\") " pod="openshift-monitoring/node-exporter-szhn4"
Oct 31 21:16:18.761009 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.760870    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d98a0acd-99d1-498f-a483-12ea20459971-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-kl9j9\" (UID: \"d98a0acd-99d1-498f-a483-12ea20459971\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-kl9j9"
Oct 31 21:16:18.761009 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.760885    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqmnk\" (UniqueName: \"kubernetes.io/projected/d98a0acd-99d1-498f-a483-12ea20459971-kube-api-access-xqmnk\") pod \"openshift-state-metrics-7bd7489957-kl9j9\" (UID: \"d98a0acd-99d1-498f-a483-12ea20459971\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-kl9j9"
Oct 31 21:16:18.761096 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.761064    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d7030dce-004e-405d-8789-014516345b75-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-szhn4\" (UID: \"d7030dce-004e-405d-8789-014516345b75\") " pod="openshift-monitoring/node-exporter-szhn4"
Oct 31 21:16:18.762812 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.762788    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d98a0acd-99d1-498f-a483-12ea20459971-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-kl9j9\" (UID: \"d98a0acd-99d1-498f-a483-12ea20459971\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-kl9j9"
Oct 31 21:16:18.762918 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.762873    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d98a0acd-99d1-498f-a483-12ea20459971-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-kl9j9\" (UID: \"d98a0acd-99d1-498f-a483-12ea20459971\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-kl9j9"
Oct 31 21:16:18.775537 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.775510    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:18.785501 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.785454    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqmnk\" (UniqueName: \"kubernetes.io/projected/d98a0acd-99d1-498f-a483-12ea20459971-kube-api-access-xqmnk\") pod \"openshift-state-metrics-7bd7489957-kl9j9\" (UID: \"d98a0acd-99d1-498f-a483-12ea20459971\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-kl9j9"
Oct 31 21:16:18.845972 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.845926    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-kl9j9"
Oct 31 21:16:18.862268 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.862239    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d7030dce-004e-405d-8789-014516345b75-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-szhn4\" (UID: \"d7030dce-004e-405d-8789-014516345b75\") " pod="openshift-monitoring/node-exporter-szhn4"
Oct 31 21:16:18.862394 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.862275    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e31eea5-fbe9-47c4-98fa-f2e9af6239e4-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-r7cc2\" (UID: \"0e31eea5-fbe9-47c4-98fa-f2e9af6239e4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-r7cc2"
Oct 31 21:16:18.862394 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.862299    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d7030dce-004e-405d-8789-014516345b75-node-exporter-textfile\") pod \"node-exporter-szhn4\" (UID: \"d7030dce-004e-405d-8789-014516345b75\") " pod="openshift-monitoring/node-exporter-szhn4"
Oct 31 21:16:18.862394 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.862318    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0e31eea5-fbe9-47c4-98fa-f2e9af6239e4-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-r7cc2\" (UID: \"0e31eea5-fbe9-47c4-98fa-f2e9af6239e4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-r7cc2"
Oct 31 21:16:18.862394 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.862348    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d7030dce-004e-405d-8789-014516345b75-sys\") pod \"node-exporter-szhn4\" (UID: \"d7030dce-004e-405d-8789-014516345b75\") " pod="openshift-monitoring/node-exporter-szhn4"
Oct 31 21:16:18.862394 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.862375    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d7030dce-004e-405d-8789-014516345b75-root\") pod \"node-exporter-szhn4\" (UID: \"d7030dce-004e-405d-8789-014516345b75\") " pod="openshift-monitoring/node-exporter-szhn4"
Oct 31 21:16:18.862626 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.862409    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7030dce-004e-405d-8789-014516345b75-metrics-client-ca\") pod \"node-exporter-szhn4\" (UID: \"d7030dce-004e-405d-8789-014516345b75\") " pod="openshift-monitoring/node-exporter-szhn4"
Oct 31 21:16:18.862626 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.862430    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nld6t\" (UniqueName: \"kubernetes.io/projected/d7030dce-004e-405d-8789-014516345b75-kube-api-access-nld6t\") pod \"node-exporter-szhn4\" (UID: \"d7030dce-004e-405d-8789-014516345b75\") " pod="openshift-monitoring/node-exporter-szhn4"
Oct 31 21:16:18.863137 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.863108    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d7030dce-004e-405d-8789-014516345b75-node-exporter-textfile\") pod \"node-exporter-szhn4\" (UID: \"d7030dce-004e-405d-8789-014516345b75\") " pod="openshift-monitoring/node-exporter-szhn4"
Oct 31 21:16:18.864487 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.863442    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7030dce-004e-405d-8789-014516345b75-metrics-client-ca\") pod \"node-exporter-szhn4\" (UID: \"d7030dce-004e-405d-8789-014516345b75\") " pod="openshift-monitoring/node-exporter-szhn4"
Oct 31 21:16:18.864487 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.862458    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlcp2\" (UniqueName: \"kubernetes.io/projected/0e31eea5-fbe9-47c4-98fa-f2e9af6239e4-kube-api-access-qlcp2\") pod \"kube-state-metrics-5dc8d7d89b-r7cc2\" (UID: \"0e31eea5-fbe9-47c4-98fa-f2e9af6239e4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-r7cc2"
Oct 31 21:16:18.864487 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.863537    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e31eea5-fbe9-47c4-98fa-f2e9af6239e4-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-r7cc2\" (UID: \"0e31eea5-fbe9-47c4-98fa-f2e9af6239e4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-r7cc2"
Oct 31 21:16:18.864487 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.863552    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e31eea5-fbe9-47c4-98fa-f2e9af6239e4-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-r7cc2\" (UID: \"0e31eea5-fbe9-47c4-98fa-f2e9af6239e4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-r7cc2"
Oct 31 21:16:18.864487 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.863575    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0e31eea5-fbe9-47c4-98fa-f2e9af6239e4-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-r7cc2\" (UID: \"0e31eea5-fbe9-47c4-98fa-f2e9af6239e4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-r7cc2"
Oct 31 21:16:18.864487 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.863611    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e31eea5-fbe9-47c4-98fa-f2e9af6239e4-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-r7cc2\" (UID: \"0e31eea5-fbe9-47c4-98fa-f2e9af6239e4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-r7cc2"
Oct 31 21:16:18.864487 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.863646    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d7030dce-004e-405d-8789-014516345b75-node-exporter-tls\") pod \"node-exporter-szhn4\" (UID: \"d7030dce-004e-405d-8789-014516345b75\") " pod="openshift-monitoring/node-exporter-szhn4"
Oct 31 21:16:18.864487 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.863753    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d7030dce-004e-405d-8789-014516345b75-node-exporter-accelerators-collector-config\") pod \"node-exporter-szhn4\" (UID: \"d7030dce-004e-405d-8789-014516345b75\") " pod="openshift-monitoring/node-exporter-szhn4"
Oct 31 21:16:18.864487 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.863788    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d7030dce-004e-405d-8789-014516345b75-node-exporter-wtmp\") pod \"node-exporter-szhn4\" (UID: \"d7030dce-004e-405d-8789-014516345b75\") " pod="openshift-monitoring/node-exporter-szhn4"
Oct 31 21:16:18.864487 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.863868    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0e31eea5-fbe9-47c4-98fa-f2e9af6239e4-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-r7cc2\" (UID: \"0e31eea5-fbe9-47c4-98fa-f2e9af6239e4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-r7cc2"
Oct 31 21:16:18.864487 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.862495    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d7030dce-004e-405d-8789-014516345b75-sys\") pod \"node-exporter-szhn4\" (UID: \"d7030dce-004e-405d-8789-014516345b75\") " pod="openshift-monitoring/node-exporter-szhn4"
Oct 31 21:16:18.864487 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.864115    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0e31eea5-fbe9-47c4-98fa-f2e9af6239e4-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-r7cc2\" (UID: \"0e31eea5-fbe9-47c4-98fa-f2e9af6239e4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-r7cc2"
Oct 31 21:16:18.865437 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.864708    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d7030dce-004e-405d-8789-014516345b75-node-exporter-accelerators-collector-config\") pod \"node-exporter-szhn4\" (UID: \"d7030dce-004e-405d-8789-014516345b75\") " pod="openshift-monitoring/node-exporter-szhn4"
Oct 31 21:16:18.867476 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.866213    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d7030dce-004e-405d-8789-014516345b75-root\") pod \"node-exporter-szhn4\" (UID: \"d7030dce-004e-405d-8789-014516345b75\") " pod="openshift-monitoring/node-exporter-szhn4"
Oct 31 21:16:18.867476 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.866366    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d7030dce-004e-405d-8789-014516345b75-node-exporter-wtmp\") pod \"node-exporter-szhn4\" (UID: \"d7030dce-004e-405d-8789-014516345b75\") " pod="openshift-monitoring/node-exporter-szhn4"
Oct 31 21:16:18.867476 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.866442    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e31eea5-fbe9-47c4-98fa-f2e9af6239e4-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-r7cc2\" (UID: \"0e31eea5-fbe9-47c4-98fa-f2e9af6239e4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-r7cc2"
Oct 31 21:16:18.867695 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.867645    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d7030dce-004e-405d-8789-014516345b75-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-szhn4\" (UID: \"d7030dce-004e-405d-8789-014516345b75\") " pod="openshift-monitoring/node-exporter-szhn4"
Oct 31 21:16:18.868241 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.868219    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e31eea5-fbe9-47c4-98fa-f2e9af6239e4-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-r7cc2\" (UID: \"0e31eea5-fbe9-47c4-98fa-f2e9af6239e4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-r7cc2"
Oct 31 21:16:18.869558 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.869539    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d7030dce-004e-405d-8789-014516345b75-node-exporter-tls\") pod \"node-exporter-szhn4\" (UID: \"d7030dce-004e-405d-8789-014516345b75\") " pod="openshift-monitoring/node-exporter-szhn4"
Oct 31 21:16:18.887886 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.887850    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlcp2\" (UniqueName: \"kubernetes.io/projected/0e31eea5-fbe9-47c4-98fa-f2e9af6239e4-kube-api-access-qlcp2\") pod \"kube-state-metrics-5dc8d7d89b-r7cc2\" (UID: \"0e31eea5-fbe9-47c4-98fa-f2e9af6239e4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-r7cc2"
Oct 31 21:16:18.888025 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.887997    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nld6t\" (UniqueName: \"kubernetes.io/projected/d7030dce-004e-405d-8789-014516345b75-kube-api-access-nld6t\") pod \"node-exporter-szhn4\" (UID: \"d7030dce-004e-405d-8789-014516345b75\") " pod="openshift-monitoring/node-exporter-szhn4"
Oct 31 21:16:18.891741 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.891714    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:18.912377 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:18.912104    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-szhn4"
Oct 31 21:16:18.923312 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:18.923282    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7030dce_004e_405d_8789_014516345b75.slice/crio-103bd7fedbd3260674927bcc940768243e4c828fe5e0bd627e1567a8ecea060a WatchSource:0}: Error finding container 103bd7fedbd3260674927bcc940768243e4c828fe5e0bd627e1567a8ecea060a: Status 404 returned error can't find the container with id 103bd7fedbd3260674927bcc940768243e4c828fe5e0bd627e1567a8ecea060a
Oct 31 21:16:19.000596 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:19.000560    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:19.112215 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:19.112132    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:19.159619 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:19.159586    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-kl9j9"]
Oct 31 21:16:19.162550 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:19.162525    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd98a0acd_99d1_498f_a483_12ea20459971.slice/crio-262e80b465a8adf8beabafdd56e35ac144226d352da9827831a0f0af1575ddd8 WatchSource:0}: Error finding container 262e80b465a8adf8beabafdd56e35ac144226d352da9827831a0f0af1575ddd8: Status 404 returned error can't find the container with id 262e80b465a8adf8beabafdd56e35ac144226d352da9827831a0f0af1575ddd8
Oct 31 21:16:19.184650 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:19.184623    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-r7cc2"
Oct 31 21:16:19.325980 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:19.325928    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-szhn4" event={"ID":"d7030dce-004e-405d-8789-014516345b75","Type":"ContainerStarted","Data":"103bd7fedbd3260674927bcc940768243e4c828fe5e0bd627e1567a8ecea060a"}
Oct 31 21:16:19.328240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:19.328212    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-kl9j9" event={"ID":"d98a0acd-99d1-498f-a483-12ea20459971","Type":"ContainerStarted","Data":"787af18fbbb9ddd6aad5189ac3112c852afec654d2426456eb7201dedda1a81f"}
Oct 31 21:16:19.328343 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:19.328268    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-kl9j9" event={"ID":"d98a0acd-99d1-498f-a483-12ea20459971","Type":"ContainerStarted","Data":"4362cd92885545d6cf8f9b8e805c1d330082b2b672ae69614ef8c781e758c6c7"}
Oct 31 21:16:19.328343 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:19.328287    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-kl9j9" event={"ID":"d98a0acd-99d1-498f-a483-12ea20459971","Type":"ContainerStarted","Data":"262e80b465a8adf8beabafdd56e35ac144226d352da9827831a0f0af1575ddd8"}
Oct 31 21:16:19.497653 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:19.497572    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-r7cc2"]
Oct 31 21:16:19.500677 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:19.500645    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e31eea5_fbe9_47c4_98fa_f2e9af6239e4.slice/crio-6b3ae065ee6f4aca71be643b4fcfdc2ba1cc27fca8291984f990de205185acb4 WatchSource:0}: Error finding container 6b3ae065ee6f4aca71be643b4fcfdc2ba1cc27fca8291984f990de205185acb4: Status 404 returned error can't find the container with id 6b3ae065ee6f4aca71be643b4fcfdc2ba1cc27fca8291984f990de205185acb4
Oct 31 21:16:19.643026 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:19.642899    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:19.661564 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:19.661529    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4ssl9"
Oct 31 21:16:19.764365 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:19.764339    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:19.879327 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:19.879294    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:20.021429 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:20.021332    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:20.334719 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:20.334672    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-r7cc2" event={"ID":"0e31eea5-fbe9-47c4-98fa-f2e9af6239e4","Type":"ContainerStarted","Data":"6b3ae065ee6f4aca71be643b4fcfdc2ba1cc27fca8291984f990de205185acb4"}
Oct 31 21:16:20.336452 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:20.336425    4460 generic.go:356] "Generic (PLEG): container finished" podID="d7030dce-004e-405d-8789-014516345b75" containerID="6981a23d693bfaef3860f1d3eacd45dd1b00766d256776a5b352ad2a57a54363" exitCode=0
Oct 31 21:16:20.336582 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:20.336512    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-szhn4" event={"ID":"d7030dce-004e-405d-8789-014516345b75","Type":"ContainerDied","Data":"6981a23d693bfaef3860f1d3eacd45dd1b00766d256776a5b352ad2a57a54363"}
Oct 31 21:16:20.339312 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:20.339287    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-kl9j9" event={"ID":"d98a0acd-99d1-498f-a483-12ea20459971","Type":"ContainerStarted","Data":"e0df33aa3368d054ebc0d97e74cbf30e1106773cd5f30b695db5e34931a56b7c"}
Oct 31 21:16:20.413434 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:20.413403    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:20.429357 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:20.429310    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-kl9j9" podStartSLOduration=1.5111856970000002 podStartE2EDuration="2.4292953s" totalImagesPullingTime="918.109603ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:16:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:16:20.428867552 +0000 UTC m=+20.345176981" watchObservedRunningTime="2026-10-31 21:16:20.4292953 +0000 UTC m=+20.345604731"
Oct 31 21:16:20.614063 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:20.613971    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:21.344487 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:21.344441    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-szhn4" event={"ID":"d7030dce-004e-405d-8789-014516345b75","Type":"ContainerStarted","Data":"bb73d146afced33b8a774adead7cc583cd023b22f2944ae61b7e1fdec32c61d0"}
Oct 31 21:16:21.344676 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:21.344547    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-szhn4" event={"ID":"d7030dce-004e-405d-8789-014516345b75","Type":"ContainerStarted","Data":"ff23efa449b443fac8423b2b230af483cd0c325a50cf3a2eecaa5469941899ed"}
Oct 31 21:16:21.346345 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:21.346320    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-r7cc2" event={"ID":"0e31eea5-fbe9-47c4-98fa-f2e9af6239e4","Type":"ContainerStarted","Data":"cc7820a79558805a941a82cf7e48b59be752a7fa796ebd05a856ffc4bb12ad8d"}
Oct 31 21:16:21.346464 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:21.346375    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-r7cc2" event={"ID":"0e31eea5-fbe9-47c4-98fa-f2e9af6239e4","Type":"ContainerStarted","Data":"f0e466a8e373333f08c9afff71802aa0b94e69acfa85dc7fb276bc1630f8ad5a"}
Oct 31 21:16:21.346464 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:21.346402    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-r7cc2" event={"ID":"0e31eea5-fbe9-47c4-98fa-f2e9af6239e4","Type":"ContainerStarted","Data":"b6cc5744f1f182a6979fd44c0a5dbda137ee5c29956188de1b4cf33f73d0456c"}
Oct 31 21:16:21.392232 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:21.392180    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-szhn4" podStartSLOduration=2.471822081 podStartE2EDuration="3.392167032s" totalImagesPullingTime="839.99026ms" totalInitContainerRuntime="80.354691ms" isStatefulPod=true podCreationTimestamp="2026-10-31 21:16:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:16:21.392023785 +0000 UTC m=+21.308333215" watchObservedRunningTime="2026-10-31 21:16:21.392167032 +0000 UTC m=+21.308476461"
Oct 31 21:16:21.438609 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:21.438557    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-r7cc2" podStartSLOduration=2.189688157 podStartE2EDuration="3.438542648s" totalImagesPullingTime="1.248854491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:16:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:16:21.437910553 +0000 UTC m=+21.354219982" watchObservedRunningTime="2026-10-31 21:16:21.438542648 +0000 UTC m=+21.354852073"
Oct 31 21:16:21.451423 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:16:21.451390    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:16:21.925684 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:21.925646    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:22.033991 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:22.033957    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:22.147147 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:22.147112    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:22.266108 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:22.266070    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:23.425218 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:23.425178    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-pnmj4"]
Oct 31 21:16:23.425709 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:23.425468    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:23.428269 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:23.428252    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-pnmj4"
Oct 31 21:16:23.454630 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:23.454606    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-pnmj4"]
Oct 31 21:16:23.459639 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:23.459622    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-jxhlq\""
Oct 31 21:16:23.459894 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:23.459878    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 31 21:16:23.563632 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:23.563605    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:23.600553 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:23.600526    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/123bbc02-efa9-4e3d-a00a-7e8c5f35afdc-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-pnmj4\" (UID: \"123bbc02-efa9-4e3d-a00a-7e8c5f35afdc\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-pnmj4"
Oct 31 21:16:23.600673 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:23.600566    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jdkv\" (UniqueName: \"kubernetes.io/projected/123bbc02-efa9-4e3d-a00a-7e8c5f35afdc-kube-api-access-9jdkv\") pod \"monitoring-plugin-77f4f76594-pnmj4\" (UID: \"123bbc02-efa9-4e3d-a00a-7e8c5f35afdc\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-pnmj4"
Oct 31 21:16:23.701095 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:23.701014    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jdkv\" (UniqueName: \"kubernetes.io/projected/123bbc02-efa9-4e3d-a00a-7e8c5f35afdc-kube-api-access-9jdkv\") pod \"monitoring-plugin-77f4f76594-pnmj4\" (UID: \"123bbc02-efa9-4e3d-a00a-7e8c5f35afdc\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-pnmj4"
Oct 31 21:16:23.701095 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:23.701070    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/123bbc02-efa9-4e3d-a00a-7e8c5f35afdc-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-pnmj4\" (UID: \"123bbc02-efa9-4e3d-a00a-7e8c5f35afdc\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-pnmj4"
Oct 31 21:16:23.703382 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:23.703358    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/123bbc02-efa9-4e3d-a00a-7e8c5f35afdc-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-pnmj4\" (UID: \"123bbc02-efa9-4e3d-a00a-7e8c5f35afdc\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-pnmj4"
Oct 31 21:16:23.726327 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:23.726301    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jdkv\" (UniqueName: \"kubernetes.io/projected/123bbc02-efa9-4e3d-a00a-7e8c5f35afdc-kube-api-access-9jdkv\") pod \"monitoring-plugin-77f4f76594-pnmj4\" (UID: \"123bbc02-efa9-4e3d-a00a-7e8c5f35afdc\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-pnmj4"
Oct 31 21:16:23.739062 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:23.739042    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-pnmj4"
Oct 31 21:16:23.888739 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:23.888701    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7c59959cf4-cgt62"]
Oct 31 21:16:23.893968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:23.893932    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7c59959cf4-cgt62"
Oct 31 21:16:23.902811 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:23.902788    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/e2461106-96c0-4098-a3e3-d1e71f4ff17c-telemeter-client-tls\") pod \"telemeter-client-7c59959cf4-cgt62\" (UID: \"e2461106-96c0-4098-a3e3-d1e71f4ff17c\") " pod="openshift-monitoring/telemeter-client-7c59959cf4-cgt62"
Oct 31 21:16:23.902925 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:23.902816    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2461106-96c0-4098-a3e3-d1e71f4ff17c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7c59959cf4-cgt62\" (UID: \"e2461106-96c0-4098-a3e3-d1e71f4ff17c\") " pod="openshift-monitoring/telemeter-client-7c59959cf4-cgt62"
Oct 31 21:16:23.902925 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:23.902837    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2461106-96c0-4098-a3e3-d1e71f4ff17c-metrics-client-ca\") pod \"telemeter-client-7c59959cf4-cgt62\" (UID: \"e2461106-96c0-4098-a3e3-d1e71f4ff17c\") " pod="openshift-monitoring/telemeter-client-7c59959cf4-cgt62"
Oct 31 21:16:23.902925 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:23.902868    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/e2461106-96c0-4098-a3e3-d1e71f4ff17c-federate-client-tls\") pod \"telemeter-client-7c59959cf4-cgt62\" (UID: \"e2461106-96c0-4098-a3e3-d1e71f4ff17c\") " pod="openshift-monitoring/telemeter-client-7c59959cf4-cgt62"
Oct 31 21:16:23.902925 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:23.902886    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/e2461106-96c0-4098-a3e3-d1e71f4ff17c-secret-telemeter-client\") pod \"telemeter-client-7c59959cf4-cgt62\" (UID: \"e2461106-96c0-4098-a3e3-d1e71f4ff17c\") " pod="openshift-monitoring/telemeter-client-7c59959cf4-cgt62"
Oct 31 21:16:23.902925 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:23.902902    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkkmh\" (UniqueName: \"kubernetes.io/projected/e2461106-96c0-4098-a3e3-d1e71f4ff17c-kube-api-access-nkkmh\") pod \"telemeter-client-7c59959cf4-cgt62\" (UID: \"e2461106-96c0-4098-a3e3-d1e71f4ff17c\") " pod="openshift-monitoring/telemeter-client-7c59959cf4-cgt62"
Oct 31 21:16:23.903237 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:23.902926    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e2461106-96c0-4098-a3e3-d1e71f4ff17c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7c59959cf4-cgt62\" (UID: \"e2461106-96c0-4098-a3e3-d1e71f4ff17c\") " pod="openshift-monitoring/telemeter-client-7c59959cf4-cgt62"
Oct 31 21:16:23.903237 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:23.902964    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2461106-96c0-4098-a3e3-d1e71f4ff17c-serving-certs-ca-bundle\") pod \"telemeter-client-7c59959cf4-cgt62\" (UID: \"e2461106-96c0-4098-a3e3-d1e71f4ff17c\") " pod="openshift-monitoring/telemeter-client-7c59959cf4-cgt62"
Oct 31 21:16:23.911627 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:23.911571    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Oct 31 21:16:23.930598 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:23.930568    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-4smmk\""
Oct 31 21:16:23.930745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:23.930600    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Oct 31 21:16:23.930745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:23.930722    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Oct 31 21:16:23.930844 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:23.930776    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Oct 31 21:16:23.930844 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:23.930568    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Oct 31 21:16:23.931854 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:23.931814    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7c59959cf4-cgt62"]
Oct 31 21:16:23.946106 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:23.944821    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Oct 31 21:16:24.003898 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:24.003816    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2461106-96c0-4098-a3e3-d1e71f4ff17c-metrics-client-ca\") pod \"telemeter-client-7c59959cf4-cgt62\" (UID: \"e2461106-96c0-4098-a3e3-d1e71f4ff17c\") " pod="openshift-monitoring/telemeter-client-7c59959cf4-cgt62"
Oct 31 21:16:24.003898 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:24.003890    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/e2461106-96c0-4098-a3e3-d1e71f4ff17c-federate-client-tls\") pod \"telemeter-client-7c59959cf4-cgt62\" (UID: \"e2461106-96c0-4098-a3e3-d1e71f4ff17c\") " pod="openshift-monitoring/telemeter-client-7c59959cf4-cgt62"
Oct 31 21:16:24.004113 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:24.003920    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/e2461106-96c0-4098-a3e3-d1e71f4ff17c-secret-telemeter-client\") pod \"telemeter-client-7c59959cf4-cgt62\" (UID: \"e2461106-96c0-4098-a3e3-d1e71f4ff17c\") " pod="openshift-monitoring/telemeter-client-7c59959cf4-cgt62"
Oct 31 21:16:24.004113 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:24.003964    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkkmh\" (UniqueName: \"kubernetes.io/projected/e2461106-96c0-4098-a3e3-d1e71f4ff17c-kube-api-access-nkkmh\") pod \"telemeter-client-7c59959cf4-cgt62\" (UID: \"e2461106-96c0-4098-a3e3-d1e71f4ff17c\") " pod="openshift-monitoring/telemeter-client-7c59959cf4-cgt62"
Oct 31 21:16:24.004113 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:24.004007    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e2461106-96c0-4098-a3e3-d1e71f4ff17c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7c59959cf4-cgt62\" (UID: \"e2461106-96c0-4098-a3e3-d1e71f4ff17c\") " pod="openshift-monitoring/telemeter-client-7c59959cf4-cgt62"
Oct 31 21:16:24.004113 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:24.004038    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2461106-96c0-4098-a3e3-d1e71f4ff17c-serving-certs-ca-bundle\") pod \"telemeter-client-7c59959cf4-cgt62\" (UID: \"e2461106-96c0-4098-a3e3-d1e71f4ff17c\") " pod="openshift-monitoring/telemeter-client-7c59959cf4-cgt62"
Oct 31 21:16:24.004113 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:24.004095    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/e2461106-96c0-4098-a3e3-d1e71f4ff17c-telemeter-client-tls\") pod \"telemeter-client-7c59959cf4-cgt62\" (UID: \"e2461106-96c0-4098-a3e3-d1e71f4ff17c\") " pod="openshift-monitoring/telemeter-client-7c59959cf4-cgt62"
Oct 31 21:16:24.004369 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:24.004148    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2461106-96c0-4098-a3e3-d1e71f4ff17c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7c59959cf4-cgt62\" (UID: \"e2461106-96c0-4098-a3e3-d1e71f4ff17c\") " pod="openshift-monitoring/telemeter-client-7c59959cf4-cgt62"
Oct 31 21:16:24.005091 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:24.005055    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e2461106-96c0-4098-a3e3-d1e71f4ff17c-metrics-client-ca\") pod \"telemeter-client-7c59959cf4-cgt62\" (UID: \"e2461106-96c0-4098-a3e3-d1e71f4ff17c\") " pod="openshift-monitoring/telemeter-client-7c59959cf4-cgt62"
Oct 31 21:16:24.005200 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:24.005150    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2461106-96c0-4098-a3e3-d1e71f4ff17c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7c59959cf4-cgt62\" (UID: \"e2461106-96c0-4098-a3e3-d1e71f4ff17c\") " pod="openshift-monitoring/telemeter-client-7c59959cf4-cgt62"
Oct 31 21:16:24.005200 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:24.005175    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2461106-96c0-4098-a3e3-d1e71f4ff17c-serving-certs-ca-bundle\") pod \"telemeter-client-7c59959cf4-cgt62\" (UID: \"e2461106-96c0-4098-a3e3-d1e71f4ff17c\") " pod="openshift-monitoring/telemeter-client-7c59959cf4-cgt62"
Oct 31 21:16:24.006492 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:24.006437    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e2461106-96c0-4098-a3e3-d1e71f4ff17c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7c59959cf4-cgt62\" (UID: \"e2461106-96c0-4098-a3e3-d1e71f4ff17c\") " pod="openshift-monitoring/telemeter-client-7c59959cf4-cgt62"
Oct 31 21:16:24.006646 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:24.006627    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/e2461106-96c0-4098-a3e3-d1e71f4ff17c-secret-telemeter-client\") pod \"telemeter-client-7c59959cf4-cgt62\" (UID: \"e2461106-96c0-4098-a3e3-d1e71f4ff17c\") " pod="openshift-monitoring/telemeter-client-7c59959cf4-cgt62"
Oct 31 21:16:24.006929 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:24.006904    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/e2461106-96c0-4098-a3e3-d1e71f4ff17c-telemeter-client-tls\") pod \"telemeter-client-7c59959cf4-cgt62\" (UID: \"e2461106-96c0-4098-a3e3-d1e71f4ff17c\") " pod="openshift-monitoring/telemeter-client-7c59959cf4-cgt62"
Oct 31 21:16:24.006929 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:24.006920    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/e2461106-96c0-4098-a3e3-d1e71f4ff17c-federate-client-tls\") pod \"telemeter-client-7c59959cf4-cgt62\" (UID: \"e2461106-96c0-4098-a3e3-d1e71f4ff17c\") " pod="openshift-monitoring/telemeter-client-7c59959cf4-cgt62"
Oct 31 21:16:24.033408 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:24.033379    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-pnmj4"]
Oct 31 21:16:24.033646 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:24.033625    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkkmh\" (UniqueName: \"kubernetes.io/projected/e2461106-96c0-4098-a3e3-d1e71f4ff17c-kube-api-access-nkkmh\") pod \"telemeter-client-7c59959cf4-cgt62\" (UID: \"e2461106-96c0-4098-a3e3-d1e71f4ff17c\") " pod="openshift-monitoring/telemeter-client-7c59959cf4-cgt62"
Oct 31 21:16:24.036266 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:24.036243    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod123bbc02_efa9_4e3d_a00a_7e8c5f35afdc.slice/crio-27fd9a977ebacd7b3d99c830df58cd028a2172fcf2832ef85ba633a0f8bab3b8 WatchSource:0}: Error finding container 27fd9a977ebacd7b3d99c830df58cd028a2172fcf2832ef85ba633a0f8bab3b8: Status 404 returned error can't find the container with id 27fd9a977ebacd7b3d99c830df58cd028a2172fcf2832ef85ba633a0f8bab3b8
Oct 31 21:16:24.213528 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:24.213496    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7c59959cf4-cgt62"
Oct 31 21:16:24.359322 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:24.359290    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-pnmj4" event={"ID":"123bbc02-efa9-4e3d-a00a-7e8c5f35afdc","Type":"ContainerStarted","Data":"27fd9a977ebacd7b3d99c830df58cd028a2172fcf2832ef85ba633a0f8bab3b8"}
Oct 31 21:16:24.567491 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:24.567407    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7c59959cf4-cgt62"]
Oct 31 21:16:24.571053 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:24.571022    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2461106_96c0_4098_a3e3_d1e71f4ff17c.slice/crio-7bf87db8646fd7c915611bb84ceadae94e127bfe3ef4ccc449ac6e29ed451d84 WatchSource:0}: Error finding container 7bf87db8646fd7c915611bb84ceadae94e127bfe3ef4ccc449ac6e29ed451d84: Status 404 returned error can't find the container with id 7bf87db8646fd7c915611bb84ceadae94e127bfe3ef4ccc449ac6e29ed451d84
Oct 31 21:16:25.368256 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:25.368150    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c59959cf4-cgt62" event={"ID":"e2461106-96c0-4098-a3e3-d1e71f4ff17c","Type":"ContainerStarted","Data":"7bf87db8646fd7c915611bb84ceadae94e127bfe3ef4ccc449ac6e29ed451d84"}
Oct 31 21:16:25.648716 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:25.648627    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:26.374793 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:26.374759    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c59959cf4-cgt62" event={"ID":"e2461106-96c0-4098-a3e3-d1e71f4ff17c","Type":"ContainerStarted","Data":"b6e4b7f2ffb13b849bf50887bf5ba62bb8e361818fb0fe05b5bd3f6135e0b4f4"}
Oct 31 21:16:26.376362 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:26.376336    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-pnmj4" event={"ID":"123bbc02-efa9-4e3d-a00a-7e8c5f35afdc","Type":"ContainerStarted","Data":"8931d2a76840555242b505262b528e919b226a254cd2c65bcb84d1183681c389"}
Oct 31 21:16:26.376614 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:26.376569    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-pnmj4"
Oct 31 21:16:26.382310 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:26.382288    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-pnmj4"
Oct 31 21:16:26.454364 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:26.454313    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-pnmj4" podStartSLOduration=2.033594244 podStartE2EDuration="3.454299853s" totalImagesPullingTime="1.420705609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:16:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:16:26.453926242 +0000 UTC m=+26.370235673" watchObservedRunningTime="2026-10-31 21:16:26.454299853 +0000 UTC m=+26.370609283"
Oct 31 21:16:26.510398 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:26.510369    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4qsqm"
Oct 31 21:16:27.381843 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:27.381803    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c59959cf4-cgt62" event={"ID":"e2461106-96c0-4098-a3e3-d1e71f4ff17c","Type":"ContainerStarted","Data":"2b40f2ae37b739dd181397ee219c7df916dc57547fa86f0682066932144c77e5"}
Oct 31 21:16:27.382306 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:27.381861    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7c59959cf4-cgt62" event={"ID":"e2461106-96c0-4098-a3e3-d1e71f4ff17c","Type":"ContainerStarted","Data":"76b4d15451d3129797087b734bc5ae9273b7a345bccd3ac84c5c54ded3daf825"}
Oct 31 21:16:27.461074 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:27.460976    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7c59959cf4-cgt62" podStartSLOduration=1.906090835 podStartE2EDuration="4.460959752s" totalImagesPullingTime="2.554868917s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:16:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:16:27.446725345 +0000 UTC m=+27.363034774" watchObservedRunningTime="2026-10-31 21:16:27.460959752 +0000 UTC m=+27.377269195"
Oct 31 21:16:28.240233 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:28.240166    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:28.454548 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:28.454493    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:28.591275 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:28.591241    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:28.777714 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:28.777677    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:29.538718 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:29.538688    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6dd789c8b7-bwnpt"]
Oct 31 21:16:29.543996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:29.543978    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-bwnpt"
Oct 31 21:16:29.571139 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:29.571107    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 31 21:16:29.571321 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:29.571306    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 31 21:16:29.603341 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:29.603306    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-zjbrd\""
Oct 31 21:16:29.647636 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:29.647602    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gftv\" (UniqueName: \"kubernetes.io/projected/b6346fef-5922-4ddc-a9dd-d802eeaf29ce-kube-api-access-8gftv\") pod \"downloads-6dd789c8b7-bwnpt\" (UID: \"b6346fef-5922-4ddc-a9dd-d802eeaf29ce\") " pod="openshift-console/downloads-6dd789c8b7-bwnpt"
Oct 31 21:16:29.647842 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:29.647655    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6346fef-5922-4ddc-a9dd-d802eeaf29ce-tmp\") pod \"downloads-6dd789c8b7-bwnpt\" (UID: \"b6346fef-5922-4ddc-a9dd-d802eeaf29ce\") " pod="openshift-console/downloads-6dd789c8b7-bwnpt"
Oct 31 21:16:29.661031 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:29.660998    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-bwnpt"]
Oct 31 21:16:29.748765 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:29.748734    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gftv\" (UniqueName: \"kubernetes.io/projected/b6346fef-5922-4ddc-a9dd-d802eeaf29ce-kube-api-access-8gftv\") pod \"downloads-6dd789c8b7-bwnpt\" (UID: \"b6346fef-5922-4ddc-a9dd-d802eeaf29ce\") " pod="openshift-console/downloads-6dd789c8b7-bwnpt"
Oct 31 21:16:29.748986 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:29.748801    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6346fef-5922-4ddc-a9dd-d802eeaf29ce-tmp\") pod \"downloads-6dd789c8b7-bwnpt\" (UID: \"b6346fef-5922-4ddc-a9dd-d802eeaf29ce\") " pod="openshift-console/downloads-6dd789c8b7-bwnpt"
Oct 31 21:16:29.749207 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:29.749186    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6346fef-5922-4ddc-a9dd-d802eeaf29ce-tmp\") pod \"downloads-6dd789c8b7-bwnpt\" (UID: \"b6346fef-5922-4ddc-a9dd-d802eeaf29ce\") " pod="openshift-console/downloads-6dd789c8b7-bwnpt"
Oct 31 21:16:29.777680 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:29.777644    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gftv\" (UniqueName: \"kubernetes.io/projected/b6346fef-5922-4ddc-a9dd-d802eeaf29ce-kube-api-access-8gftv\") pod \"downloads-6dd789c8b7-bwnpt\" (UID: \"b6346fef-5922-4ddc-a9dd-d802eeaf29ce\") " pod="openshift-console/downloads-6dd789c8b7-bwnpt"
Oct 31 21:16:29.857253 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:29.857001    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-bwnpt"
Oct 31 21:16:30.041291 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:30.041260    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:30.163049 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:30.162969    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:30.208303 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:30.208274    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-bwnpt"]
Oct 31 21:16:30.211316 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:30.211292    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6346fef_5922_4ddc_a9dd_d802eeaf29ce.slice/crio-ce2ddf25bb4fe635cac02f0f4dc3562e36640d408849b35147ad4eb3980d178e WatchSource:0}: Error finding container ce2ddf25bb4fe635cac02f0f4dc3562e36640d408849b35147ad4eb3980d178e: Status 404 returned error can't find the container with id ce2ddf25bb4fe635cac02f0f4dc3562e36640d408849b35147ad4eb3980d178e
Oct 31 21:16:30.367866 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:30.367831    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:30.392240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:30.392203    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-bwnpt" event={"ID":"b6346fef-5922-4ddc-a9dd-d802eeaf29ce","Type":"ContainerStarted","Data":"ce2ddf25bb4fe635cac02f0f4dc3562e36640d408849b35147ad4eb3980d178e"}
Oct 31 21:16:30.488268 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:30.488181    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:31.274559 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:31.274525    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:31.388481 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:31.388446    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:32.455211 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:32.455162    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:32.585769 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:32.585722    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:33.814438 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:33.814404    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-6cdc5ffcbb-fsrkr"]
Oct 31 21:16:33.837743 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:33.837711    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cdc5ffcbb-fsrkr"
Oct 31 21:16:33.839600 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:33.839563    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cdc5ffcbb-fsrkr"]
Oct 31 21:16:33.854238 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:33.854217    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 31 21:16:33.867653 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:33.867629    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 31 21:16:33.867768 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:33.867629    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 31 21:16:33.867768 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:33.867634    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 31 21:16:33.867768 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:33.867682    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-j6cmx\""
Oct 31 21:16:33.867768 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:33.867700    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 31 21:16:33.887705 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:33.887665    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/81680626-b8d2-4dca-904a-7823a924ac88-console-serving-cert\") pod \"console-6cdc5ffcbb-fsrkr\" (UID: \"81680626-b8d2-4dca-904a-7823a924ac88\") " pod="openshift-console/console-6cdc5ffcbb-fsrkr"
Oct 31 21:16:33.887845 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:33.887755    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fvn2\" (UniqueName: \"kubernetes.io/projected/81680626-b8d2-4dca-904a-7823a924ac88-kube-api-access-6fvn2\") pod \"console-6cdc5ffcbb-fsrkr\" (UID: \"81680626-b8d2-4dca-904a-7823a924ac88\") " pod="openshift-console/console-6cdc5ffcbb-fsrkr"
Oct 31 21:16:33.887845 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:33.887826    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/81680626-b8d2-4dca-904a-7823a924ac88-console-oauth-config\") pod \"console-6cdc5ffcbb-fsrkr\" (UID: \"81680626-b8d2-4dca-904a-7823a924ac88\") " pod="openshift-console/console-6cdc5ffcbb-fsrkr"
Oct 31 21:16:33.887845 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:33.887877    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/81680626-b8d2-4dca-904a-7823a924ac88-console-config\") pod \"console-6cdc5ffcbb-fsrkr\" (UID: \"81680626-b8d2-4dca-904a-7823a924ac88\") " pod="openshift-console/console-6cdc5ffcbb-fsrkr"
Oct 31 21:16:33.888218 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:33.887915    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/81680626-b8d2-4dca-904a-7823a924ac88-service-ca\") pod \"console-6cdc5ffcbb-fsrkr\" (UID: \"81680626-b8d2-4dca-904a-7823a924ac88\") " pod="openshift-console/console-6cdc5ffcbb-fsrkr"
Oct 31 21:16:33.888218 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:33.888041    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/81680626-b8d2-4dca-904a-7823a924ac88-tmp\") pod \"console-6cdc5ffcbb-fsrkr\" (UID: \"81680626-b8d2-4dca-904a-7823a924ac88\") " pod="openshift-console/console-6cdc5ffcbb-fsrkr"
Oct 31 21:16:33.888218 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:33.888117    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/81680626-b8d2-4dca-904a-7823a924ac88-oauth-serving-cert\") pod \"console-6cdc5ffcbb-fsrkr\" (UID: \"81680626-b8d2-4dca-904a-7823a924ac88\") " pod="openshift-console/console-6cdc5ffcbb-fsrkr"
Oct 31 21:16:33.988473 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:33.988439    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fvn2\" (UniqueName: \"kubernetes.io/projected/81680626-b8d2-4dca-904a-7823a924ac88-kube-api-access-6fvn2\") pod \"console-6cdc5ffcbb-fsrkr\" (UID: \"81680626-b8d2-4dca-904a-7823a924ac88\") " pod="openshift-console/console-6cdc5ffcbb-fsrkr"
Oct 31 21:16:33.988473 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:33.988478    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/81680626-b8d2-4dca-904a-7823a924ac88-console-oauth-config\") pod \"console-6cdc5ffcbb-fsrkr\" (UID: \"81680626-b8d2-4dca-904a-7823a924ac88\") " pod="openshift-console/console-6cdc5ffcbb-fsrkr"
Oct 31 21:16:33.988716 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:33.988512    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/81680626-b8d2-4dca-904a-7823a924ac88-console-config\") pod \"console-6cdc5ffcbb-fsrkr\" (UID: \"81680626-b8d2-4dca-904a-7823a924ac88\") " pod="openshift-console/console-6cdc5ffcbb-fsrkr"
Oct 31 21:16:33.988716 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:33.988641    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/81680626-b8d2-4dca-904a-7823a924ac88-service-ca\") pod \"console-6cdc5ffcbb-fsrkr\" (UID: \"81680626-b8d2-4dca-904a-7823a924ac88\") " pod="openshift-console/console-6cdc5ffcbb-fsrkr"
Oct 31 21:16:33.988840 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:33.988818    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/81680626-b8d2-4dca-904a-7823a924ac88-tmp\") pod \"console-6cdc5ffcbb-fsrkr\" (UID: \"81680626-b8d2-4dca-904a-7823a924ac88\") " pod="openshift-console/console-6cdc5ffcbb-fsrkr"
Oct 31 21:16:33.989365 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:33.988895    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/81680626-b8d2-4dca-904a-7823a924ac88-oauth-serving-cert\") pod \"console-6cdc5ffcbb-fsrkr\" (UID: \"81680626-b8d2-4dca-904a-7823a924ac88\") " pod="openshift-console/console-6cdc5ffcbb-fsrkr"
Oct 31 21:16:33.989365 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:33.988967    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/81680626-b8d2-4dca-904a-7823a924ac88-console-serving-cert\") pod \"console-6cdc5ffcbb-fsrkr\" (UID: \"81680626-b8d2-4dca-904a-7823a924ac88\") " pod="openshift-console/console-6cdc5ffcbb-fsrkr"
Oct 31 21:16:33.989365 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:33.989293    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/81680626-b8d2-4dca-904a-7823a924ac88-tmp\") pod \"console-6cdc5ffcbb-fsrkr\" (UID: \"81680626-b8d2-4dca-904a-7823a924ac88\") " pod="openshift-console/console-6cdc5ffcbb-fsrkr"
Oct 31 21:16:33.989365 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:33.989344    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/81680626-b8d2-4dca-904a-7823a924ac88-console-config\") pod \"console-6cdc5ffcbb-fsrkr\" (UID: \"81680626-b8d2-4dca-904a-7823a924ac88\") " pod="openshift-console/console-6cdc5ffcbb-fsrkr"
Oct 31 21:16:33.989592 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:33.989344    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/81680626-b8d2-4dca-904a-7823a924ac88-service-ca\") pod \"console-6cdc5ffcbb-fsrkr\" (UID: \"81680626-b8d2-4dca-904a-7823a924ac88\") " pod="openshift-console/console-6cdc5ffcbb-fsrkr"
Oct 31 21:16:33.989592 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:33.989519    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/81680626-b8d2-4dca-904a-7823a924ac88-oauth-serving-cert\") pod \"console-6cdc5ffcbb-fsrkr\" (UID: \"81680626-b8d2-4dca-904a-7823a924ac88\") " pod="openshift-console/console-6cdc5ffcbb-fsrkr"
Oct 31 21:16:33.991487 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:33.991460    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/81680626-b8d2-4dca-904a-7823a924ac88-console-oauth-config\") pod \"console-6cdc5ffcbb-fsrkr\" (UID: \"81680626-b8d2-4dca-904a-7823a924ac88\") " pod="openshift-console/console-6cdc5ffcbb-fsrkr"
Oct 31 21:16:33.991584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:33.991504    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/81680626-b8d2-4dca-904a-7823a924ac88-console-serving-cert\") pod \"console-6cdc5ffcbb-fsrkr\" (UID: \"81680626-b8d2-4dca-904a-7823a924ac88\") " pod="openshift-console/console-6cdc5ffcbb-fsrkr"
Oct 31 21:16:34.017046 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:34.017020    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fvn2\" (UniqueName: \"kubernetes.io/projected/81680626-b8d2-4dca-904a-7823a924ac88-kube-api-access-6fvn2\") pod \"console-6cdc5ffcbb-fsrkr\" (UID: \"81680626-b8d2-4dca-904a-7823a924ac88\") " pod="openshift-console/console-6cdc5ffcbb-fsrkr"
Oct 31 21:16:34.050284 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:34.050247    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:34.174599 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:34.174523    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cdc5ffcbb-fsrkr"
Oct 31 21:16:34.190414 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:34.190374    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:34.301885 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:34.301846    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:34.502851 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:34.502822    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cdc5ffcbb-fsrkr"]
Oct 31 21:16:34.505073 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:34.505047    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81680626_b8d2_4dca_904a_7823a924ac88.slice/crio-4d45f630dd9b4e61403cca84a366fb76cafd98cf3598dda21feb0dd5f1c2c652 WatchSource:0}: Error finding container 4d45f630dd9b4e61403cca84a366fb76cafd98cf3598dda21feb0dd5f1c2c652: Status 404 returned error can't find the container with id 4d45f630dd9b4e61403cca84a366fb76cafd98cf3598dda21feb0dd5f1c2c652
Oct 31 21:16:34.789582 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:34.789557    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-bsvln"
Oct 31 21:16:35.413473 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:35.413436    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cdc5ffcbb-fsrkr" event={"ID":"81680626-b8d2-4dca-904a-7823a924ac88","Type":"ContainerStarted","Data":"4d45f630dd9b4e61403cca84a366fb76cafd98cf3598dda21feb0dd5f1c2c652"}
Oct 31 21:16:38.428049 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:38.428011    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cdc5ffcbb-fsrkr" event={"ID":"81680626-b8d2-4dca-904a-7823a924ac88","Type":"ContainerStarted","Data":"8d287305d499a135ffbbd615b14330ca95859c97c2f207fb42b3d8a18d489dee"}
Oct 31 21:16:38.693895 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:38.693814    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:38.803506 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:38.803331    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:42.880964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:42.880865    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:43.018820 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:43.018781    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:44.175141 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:44.175104    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6cdc5ffcbb-fsrkr"
Oct 31 21:16:44.175578 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:44.175287    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6cdc5ffcbb-fsrkr"
Oct 31 21:16:44.180624 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:44.180601    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6cdc5ffcbb-fsrkr"
Oct 31 21:16:44.227517 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:44.227463    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-6cdc5ffcbb-fsrkr" podStartSLOduration=8.177312683 podStartE2EDuration="11.227442885s" totalImagesPullingTime="3.050130202s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:16:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:16:38.475799665 +0000 UTC m=+38.392109094" watchObservedRunningTime="2026-10-31 21:16:44.227442885 +0000 UTC m=+44.143752313"
Oct 31 21:16:44.414080 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:44.414039    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:44.454455 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:44.454383    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6cdc5ffcbb-fsrkr"
Oct 31 21:16:44.552310 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:44.552278    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:45.013952 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:45.013905    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-7675b66b5c-9ffvv"]
Oct 31 21:16:45.017801 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:45.017763    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7675b66b5c-9ffvv"
Oct 31 21:16:45.051035 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:45.051009    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7675b66b5c-9ffvv"]
Oct 31 21:16:45.078888 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:45.078859    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 31 21:16:45.086117 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:45.086089    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-service-ca\") pod \"console-7675b66b5c-9ffvv\" (UID: \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\") " pod="openshift-console/console-7675b66b5c-9ffvv"
Oct 31 21:16:45.086252 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:45.086129    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-console-serving-cert\") pod \"console-7675b66b5c-9ffvv\" (UID: \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\") " pod="openshift-console/console-7675b66b5c-9ffvv"
Oct 31 21:16:45.086252 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:45.086223    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wb6q8\" (UniqueName: \"kubernetes.io/projected/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-kube-api-access-wb6q8\") pod \"console-7675b66b5c-9ffvv\" (UID: \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\") " pod="openshift-console/console-7675b66b5c-9ffvv"
Oct 31 21:16:45.086368 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:45.086269    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-console-oauth-config\") pod \"console-7675b66b5c-9ffvv\" (UID: \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\") " pod="openshift-console/console-7675b66b5c-9ffvv"
Oct 31 21:16:45.086368 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:45.086315    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-console-config\") pod \"console-7675b66b5c-9ffvv\" (UID: \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\") " pod="openshift-console/console-7675b66b5c-9ffvv"
Oct 31 21:16:45.086368 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:45.086339    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-trusted-ca-bundle\") pod \"console-7675b66b5c-9ffvv\" (UID: \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\") " pod="openshift-console/console-7675b66b5c-9ffvv"
Oct 31 21:16:45.086368 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:45.086360    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-tmp\") pod \"console-7675b66b5c-9ffvv\" (UID: \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\") " pod="openshift-console/console-7675b66b5c-9ffvv"
Oct 31 21:16:45.086583 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:45.086379    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-oauth-serving-cert\") pod \"console-7675b66b5c-9ffvv\" (UID: \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\") " pod="openshift-console/console-7675b66b5c-9ffvv"
Oct 31 21:16:45.186773 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:45.186739    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-service-ca\") pod \"console-7675b66b5c-9ffvv\" (UID: \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\") " pod="openshift-console/console-7675b66b5c-9ffvv"
Oct 31 21:16:45.186773 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:45.186779    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-console-serving-cert\") pod \"console-7675b66b5c-9ffvv\" (UID: \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\") " pod="openshift-console/console-7675b66b5c-9ffvv"
Oct 31 21:16:45.187319 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:45.186814    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wb6q8\" (UniqueName: \"kubernetes.io/projected/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-kube-api-access-wb6q8\") pod \"console-7675b66b5c-9ffvv\" (UID: \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\") " pod="openshift-console/console-7675b66b5c-9ffvv"
Oct 31 21:16:45.187319 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:45.186860    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-console-oauth-config\") pod \"console-7675b66b5c-9ffvv\" (UID: \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\") " pod="openshift-console/console-7675b66b5c-9ffvv"
Oct 31 21:16:45.187319 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:45.186895    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-console-config\") pod \"console-7675b66b5c-9ffvv\" (UID: \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\") " pod="openshift-console/console-7675b66b5c-9ffvv"
Oct 31 21:16:45.187319 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:45.186924    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-trusted-ca-bundle\") pod \"console-7675b66b5c-9ffvv\" (UID: \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\") " pod="openshift-console/console-7675b66b5c-9ffvv"
Oct 31 21:16:45.187319 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:45.186979    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-tmp\") pod \"console-7675b66b5c-9ffvv\" (UID: \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\") " pod="openshift-console/console-7675b66b5c-9ffvv"
Oct 31 21:16:45.187319 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:45.187011    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-oauth-serving-cert\") pod \"console-7675b66b5c-9ffvv\" (UID: \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\") " pod="openshift-console/console-7675b66b5c-9ffvv"
Oct 31 21:16:45.187652 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:45.187627    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-service-ca\") pod \"console-7675b66b5c-9ffvv\" (UID: \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\") " pod="openshift-console/console-7675b66b5c-9ffvv"
Oct 31 21:16:45.188077 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:45.187713    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-oauth-serving-cert\") pod \"console-7675b66b5c-9ffvv\" (UID: \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\") " pod="openshift-console/console-7675b66b5c-9ffvv"
Oct 31 21:16:45.188077 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:45.187834    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-console-config\") pod \"console-7675b66b5c-9ffvv\" (UID: \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\") " pod="openshift-console/console-7675b66b5c-9ffvv"
Oct 31 21:16:45.188077 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:45.188029    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-tmp\") pod \"console-7675b66b5c-9ffvv\" (UID: \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\") " pod="openshift-console/console-7675b66b5c-9ffvv"
Oct 31 21:16:45.188452 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:45.188426    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-trusted-ca-bundle\") pod \"console-7675b66b5c-9ffvv\" (UID: \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\") " pod="openshift-console/console-7675b66b5c-9ffvv"
Oct 31 21:16:45.190063 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:45.190011    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-console-serving-cert\") pod \"console-7675b66b5c-9ffvv\" (UID: \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\") " pod="openshift-console/console-7675b66b5c-9ffvv"
Oct 31 21:16:45.190063 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:45.190020    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-console-oauth-config\") pod \"console-7675b66b5c-9ffvv\" (UID: \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\") " pod="openshift-console/console-7675b66b5c-9ffvv"
Oct 31 21:16:45.212295 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:45.212274    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wb6q8\" (UniqueName: \"kubernetes.io/projected/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-kube-api-access-wb6q8\") pod \"console-7675b66b5c-9ffvv\" (UID: \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\") " pod="openshift-console/console-7675b66b5c-9ffvv"
Oct 31 21:16:45.330540 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:45.330468    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7675b66b5c-9ffvv"
Oct 31 21:16:45.637497 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:45.637428    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7675b66b5c-9ffvv"]
Oct 31 21:16:45.640479 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:16:45.640447    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode00a562f_5d3f_4c5e_9be5_4d0bb4197f5b.slice/crio-5bce357e0ab5452702fd9b9811668b2d7187d26ea491c49364a658bbc28ed761 WatchSource:0}: Error finding container 5bce357e0ab5452702fd9b9811668b2d7187d26ea491c49364a658bbc28ed761: Status 404 returned error can't find the container with id 5bce357e0ab5452702fd9b9811668b2d7187d26ea491c49364a658bbc28ed761
Oct 31 21:16:46.460477 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:46.460379    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7675b66b5c-9ffvv" event={"ID":"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b","Type":"ContainerStarted","Data":"0de392c5f48a172c0f7f4f2ac6d8434671bb74f939a101de05dc2c82e3997a39"}
Oct 31 21:16:46.460477 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:46.460451    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7675b66b5c-9ffvv" event={"ID":"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b","Type":"ContainerStarted","Data":"5bce357e0ab5452702fd9b9811668b2d7187d26ea491c49364a658bbc28ed761"}
Oct 31 21:16:46.462378 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:46.462346    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-bwnpt" event={"ID":"b6346fef-5922-4ddc-a9dd-d802eeaf29ce","Type":"ContainerStarted","Data":"f3a9fa9caa9e297c67461e83fe500438bcd7b3a4e069ccc0b60a495363db4b46"}
Oct 31 21:16:46.462706 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:46.462651    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6dd789c8b7-bwnpt"
Oct 31 21:16:46.483347 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:46.483321    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6dd789c8b7-bwnpt"
Oct 31 21:16:46.508320 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:46.508262    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-7675b66b5c-9ffvv" podStartSLOduration=2.5082437029999998 podStartE2EDuration="2.508243703s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:16:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:16:46.507549445 +0000 UTC m=+46.423858876" watchObservedRunningTime="2026-10-31 21:16:46.508243703 +0000 UTC m=+46.424553139"
Oct 31 21:16:46.554855 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:46.554805    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/downloads-6dd789c8b7-bwnpt" podStartSLOduration=1.659770178 podStartE2EDuration="17.554785234s" totalImagesPullingTime="15.895015056s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:16:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:16:46.554536146 +0000 UTC m=+46.470845575" watchObservedRunningTime="2026-10-31 21:16:46.554785234 +0000 UTC m=+46.471094665"
Oct 31 21:16:49.011384 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:49.011347    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:51.455297 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:16:51.455254    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:16:53.798564 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:53.798516    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:53.915642 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:53.915602    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:55.330855 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:55.330822    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7675b66b5c-9ffvv"
Oct 31 21:16:55.331251 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:55.330875    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7675b66b5c-9ffvv"
Oct 31 21:16:55.335484 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:55.335462    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7675b66b5c-9ffvv"
Oct 31 21:16:55.501394 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:55.501367    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7675b66b5c-9ffvv"
Oct 31 21:16:55.587118 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:55.587046    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cdc5ffcbb-fsrkr"]
Oct 31 21:16:59.034226 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:59.034192    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:16:59.153698 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:16:59.153667    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:17:00.427600 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:00.427556    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:17:00.540867 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:00.540836    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:17:04.690436 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:04.690397    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:17:04.812828 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:04.812794    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:17:20.608519 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:20.608459    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-6cdc5ffcbb-fsrkr" podUID="81680626-b8d2-4dca-904a-7823a924ac88" containerName="console" containerID="cri-o://8d287305d499a135ffbbd615b14330ca95859c97c2f207fb42b3d8a18d489dee" gracePeriod=15
Oct 31 21:17:20.901237 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:20.901211    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cdc5ffcbb-fsrkr"
Oct 31 21:17:21.060694 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:21.060666    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/81680626-b8d2-4dca-904a-7823a924ac88-console-config\" (UniqueName: \"kubernetes.io/configmap/81680626-b8d2-4dca-904a-7823a924ac88-console-config\") pod \"81680626-b8d2-4dca-904a-7823a924ac88\" (UID: \"81680626-b8d2-4dca-904a-7823a924ac88\") "
Oct 31 21:17:21.060865 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:21.060725    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81680626-b8d2-4dca-904a-7823a924ac88-kube-api-access-6fvn2\" (UniqueName: \"kubernetes.io/projected/81680626-b8d2-4dca-904a-7823a924ac88-kube-api-access-6fvn2\") pod \"81680626-b8d2-4dca-904a-7823a924ac88\" (UID: \"81680626-b8d2-4dca-904a-7823a924ac88\") "
Oct 31 21:17:21.060865 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:21.060772    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/81680626-b8d2-4dca-904a-7823a924ac88-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/81680626-b8d2-4dca-904a-7823a924ac88-console-serving-cert\") pod \"81680626-b8d2-4dca-904a-7823a924ac88\" (UID: \"81680626-b8d2-4dca-904a-7823a924ac88\") "
Oct 31 21:17:21.060865 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:21.060793    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/81680626-b8d2-4dca-904a-7823a924ac88-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/81680626-b8d2-4dca-904a-7823a924ac88-console-oauth-config\") pod \"81680626-b8d2-4dca-904a-7823a924ac88\" (UID: \"81680626-b8d2-4dca-904a-7823a924ac88\") "
Oct 31 21:17:21.060865 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:21.060815    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81680626-b8d2-4dca-904a-7823a924ac88-tmp\" (UniqueName: \"kubernetes.io/empty-dir/81680626-b8d2-4dca-904a-7823a924ac88-tmp\") pod \"81680626-b8d2-4dca-904a-7823a924ac88\" (UID: \"81680626-b8d2-4dca-904a-7823a924ac88\") "
Oct 31 21:17:21.060865 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:21.060852    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/81680626-b8d2-4dca-904a-7823a924ac88-service-ca\" (UniqueName: \"kubernetes.io/configmap/81680626-b8d2-4dca-904a-7823a924ac88-service-ca\") pod \"81680626-b8d2-4dca-904a-7823a924ac88\" (UID: \"81680626-b8d2-4dca-904a-7823a924ac88\") "
Oct 31 21:17:21.061163 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:21.060876    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/81680626-b8d2-4dca-904a-7823a924ac88-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/81680626-b8d2-4dca-904a-7823a924ac88-oauth-serving-cert\") pod \"81680626-b8d2-4dca-904a-7823a924ac88\" (UID: \"81680626-b8d2-4dca-904a-7823a924ac88\") "
Oct 31 21:17:21.061218 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:21.061158    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81680626-b8d2-4dca-904a-7823a924ac88-console-config" pod "81680626-b8d2-4dca-904a-7823a924ac88" (UID: "81680626-b8d2-4dca-904a-7823a924ac88"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:17:21.061413 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:21.061390    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81680626-b8d2-4dca-904a-7823a924ac88-oauth-serving-cert" pod "81680626-b8d2-4dca-904a-7823a924ac88" (UID: "81680626-b8d2-4dca-904a-7823a924ac88"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:17:21.061496 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:21.061440    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81680626-b8d2-4dca-904a-7823a924ac88-service-ca" pod "81680626-b8d2-4dca-904a-7823a924ac88" (UID: "81680626-b8d2-4dca-904a-7823a924ac88"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:17:21.061496 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:21.061188    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81680626-b8d2-4dca-904a-7823a924ac88-tmp" pod "81680626-b8d2-4dca-904a-7823a924ac88" (UID: "81680626-b8d2-4dca-904a-7823a924ac88"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:17:21.063219 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:21.063187    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81680626-b8d2-4dca-904a-7823a924ac88-console-oauth-config" pod "81680626-b8d2-4dca-904a-7823a924ac88" (UID: "81680626-b8d2-4dca-904a-7823a924ac88"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 21:17:21.063219 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:21.063204    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81680626-b8d2-4dca-904a-7823a924ac88-kube-api-access-6fvn2" pod "81680626-b8d2-4dca-904a-7823a924ac88" (UID: "81680626-b8d2-4dca-904a-7823a924ac88"). InnerVolumeSpecName "kube-api-access-6fvn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:17:21.063350 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:21.063223    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81680626-b8d2-4dca-904a-7823a924ac88-console-serving-cert" pod "81680626-b8d2-4dca-904a-7823a924ac88" (UID: "81680626-b8d2-4dca-904a-7823a924ac88"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 21:17:21.161641 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:21.161577    4460 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/81680626-b8d2-4dca-904a-7823a924ac88-console-serving-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:17:21.161641 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:21.161602    4460 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/81680626-b8d2-4dca-904a-7823a924ac88-console-oauth-config\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:17:21.161641 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:21.161617    4460 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/81680626-b8d2-4dca-904a-7823a924ac88-tmp\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:17:21.161641 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:21.161631    4460 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/81680626-b8d2-4dca-904a-7823a924ac88-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:17:21.161840 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:21.161645    4460 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/81680626-b8d2-4dca-904a-7823a924ac88-oauth-serving-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:17:21.161840 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:21.161658    4460 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/81680626-b8d2-4dca-904a-7823a924ac88-console-config\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:17:21.161840 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:21.161673    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fvn2\" (UniqueName: \"kubernetes.io/projected/81680626-b8d2-4dca-904a-7823a924ac88-kube-api-access-6fvn2\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:17:21.451231 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:17:21.451151    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:17:21.585904 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:21.585869    4460 generic.go:356] "Generic (PLEG): container finished" podID="81680626-b8d2-4dca-904a-7823a924ac88" containerID="8d287305d499a135ffbbd615b14330ca95859c97c2f207fb42b3d8a18d489dee" exitCode=0
Oct 31 21:17:21.586087 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:21.585916    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cdc5ffcbb-fsrkr"
Oct 31 21:17:21.586087 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:21.585964    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cdc5ffcbb-fsrkr" event={"ID":"81680626-b8d2-4dca-904a-7823a924ac88","Type":"ContainerDied","Data":"8d287305d499a135ffbbd615b14330ca95859c97c2f207fb42b3d8a18d489dee"}
Oct 31 21:17:21.586087 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:21.586015    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cdc5ffcbb-fsrkr" event={"ID":"81680626-b8d2-4dca-904a-7823a924ac88","Type":"ContainerDied","Data":"4d45f630dd9b4e61403cca84a366fb76cafd98cf3598dda21feb0dd5f1c2c652"}
Oct 31 21:17:21.586087 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:21.586036    4460 scope.go:122] "RemoveContainer" containerID="8d287305d499a135ffbbd615b14330ca95859c97c2f207fb42b3d8a18d489dee"
Oct 31 21:17:21.594047 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:21.594028    4460 scope.go:122] "RemoveContainer" containerID="8d287305d499a135ffbbd615b14330ca95859c97c2f207fb42b3d8a18d489dee"
Oct 31 21:17:21.594322 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:17:21.594301    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d287305d499a135ffbbd615b14330ca95859c97c2f207fb42b3d8a18d489dee\": container with ID starting with 8d287305d499a135ffbbd615b14330ca95859c97c2f207fb42b3d8a18d489dee not found: ID does not exist" containerID="8d287305d499a135ffbbd615b14330ca95859c97c2f207fb42b3d8a18d489dee"
Oct 31 21:17:21.594372 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:21.594331    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d287305d499a135ffbbd615b14330ca95859c97c2f207fb42b3d8a18d489dee"} err="failed to get container status \"8d287305d499a135ffbbd615b14330ca95859c97c2f207fb42b3d8a18d489dee\": rpc error: code = NotFound desc = could not find container \"8d287305d499a135ffbbd615b14330ca95859c97c2f207fb42b3d8a18d489dee\": container with ID starting with 8d287305d499a135ffbbd615b14330ca95859c97c2f207fb42b3d8a18d489dee not found: ID does not exist"
Oct 31 21:17:21.640879 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:21.640856    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cdc5ffcbb-fsrkr"]
Oct 31 21:17:21.654632 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:21.654612    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6cdc5ffcbb-fsrkr"]
Oct 31 21:17:22.221922 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:22.221890    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="81680626-b8d2-4dca-904a-7823a924ac88" path="/var/lib/kubelet/pods/81680626-b8d2-4dca-904a-7823a924ac88/volumes"
Oct 31 21:17:23.702267 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:23.702233    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:17:23.816384 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:23.816346    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:17:25.499479 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:25.499439    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:17:28.287859 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.287828    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-954f87865-4q2rj"]
Oct 31 21:17:28.288358 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.288120    4460 cpu_manager.go:414] "containerMap: removing container" podUID="81680626-b8d2-4dca-904a-7823a924ac88" containerName="console"
Oct 31 21:17:28.288358 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.288132    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="81680626-b8d2-4dca-904a-7823a924ac88" containerName="console"
Oct 31 21:17:28.288358 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.288183    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="81680626-b8d2-4dca-904a-7823a924ac88" containerName="console"
Oct 31 21:17:28.313122 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.313082    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-954f87865-4q2rj"]
Oct 31 21:17:28.313319 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.313204    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-954f87865-4q2rj"
Oct 31 21:17:28.313749 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.313722    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a6becbb4-8451-4de2-b094-7867363d00ce-oauth-serving-cert\") pod \"console-954f87865-4q2rj\" (UID: \"a6becbb4-8451-4de2-b094-7867363d00ce\") " pod="openshift-console/console-954f87865-4q2rj"
Oct 31 21:17:28.314012 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.313983    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a6becbb4-8451-4de2-b094-7867363d00ce-console-config\") pod \"console-954f87865-4q2rj\" (UID: \"a6becbb4-8451-4de2-b094-7867363d00ce\") " pod="openshift-console/console-954f87865-4q2rj"
Oct 31 21:17:28.314132 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.314062    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6becbb4-8451-4de2-b094-7867363d00ce-trusted-ca-bundle\") pod \"console-954f87865-4q2rj\" (UID: \"a6becbb4-8451-4de2-b094-7867363d00ce\") " pod="openshift-console/console-954f87865-4q2rj"
Oct 31 21:17:28.314193 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.314138    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwjtf\" (UniqueName: \"kubernetes.io/projected/a6becbb4-8451-4de2-b094-7867363d00ce-kube-api-access-qwjtf\") pod \"console-954f87865-4q2rj\" (UID: \"a6becbb4-8451-4de2-b094-7867363d00ce\") " pod="openshift-console/console-954f87865-4q2rj"
Oct 31 21:17:28.314361 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.314339    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a6becbb4-8451-4de2-b094-7867363d00ce-tmp\") pod \"console-954f87865-4q2rj\" (UID: \"a6becbb4-8451-4de2-b094-7867363d00ce\") " pod="openshift-console/console-954f87865-4q2rj"
Oct 31 21:17:28.314443 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.314417    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a6becbb4-8451-4de2-b094-7867363d00ce-console-serving-cert\") pod \"console-954f87865-4q2rj\" (UID: \"a6becbb4-8451-4de2-b094-7867363d00ce\") " pod="openshift-console/console-954f87865-4q2rj"
Oct 31 21:17:28.314498 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.314461    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a6becbb4-8451-4de2-b094-7867363d00ce-service-ca\") pod \"console-954f87865-4q2rj\" (UID: \"a6becbb4-8451-4de2-b094-7867363d00ce\") " pod="openshift-console/console-954f87865-4q2rj"
Oct 31 21:17:28.314498 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.314490    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a6becbb4-8451-4de2-b094-7867363d00ce-console-oauth-config\") pod \"console-954f87865-4q2rj\" (UID: \"a6becbb4-8451-4de2-b094-7867363d00ce\") " pod="openshift-console/console-954f87865-4q2rj"
Oct 31 21:17:28.415186 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.415154    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a6becbb4-8451-4de2-b094-7867363d00ce-service-ca\") pod \"console-954f87865-4q2rj\" (UID: \"a6becbb4-8451-4de2-b094-7867363d00ce\") " pod="openshift-console/console-954f87865-4q2rj"
Oct 31 21:17:28.415186 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.415189    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a6becbb4-8451-4de2-b094-7867363d00ce-console-oauth-config\") pod \"console-954f87865-4q2rj\" (UID: \"a6becbb4-8451-4de2-b094-7867363d00ce\") " pod="openshift-console/console-954f87865-4q2rj"
Oct 31 21:17:28.415430 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.415207    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a6becbb4-8451-4de2-b094-7867363d00ce-oauth-serving-cert\") pod \"console-954f87865-4q2rj\" (UID: \"a6becbb4-8451-4de2-b094-7867363d00ce\") " pod="openshift-console/console-954f87865-4q2rj"
Oct 31 21:17:28.415430 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.415262    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a6becbb4-8451-4de2-b094-7867363d00ce-console-config\") pod \"console-954f87865-4q2rj\" (UID: \"a6becbb4-8451-4de2-b094-7867363d00ce\") " pod="openshift-console/console-954f87865-4q2rj"
Oct 31 21:17:28.415430 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.415305    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6becbb4-8451-4de2-b094-7867363d00ce-trusted-ca-bundle\") pod \"console-954f87865-4q2rj\" (UID: \"a6becbb4-8451-4de2-b094-7867363d00ce\") " pod="openshift-console/console-954f87865-4q2rj"
Oct 31 21:17:28.415430 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.415332    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwjtf\" (UniqueName: \"kubernetes.io/projected/a6becbb4-8451-4de2-b094-7867363d00ce-kube-api-access-qwjtf\") pod \"console-954f87865-4q2rj\" (UID: \"a6becbb4-8451-4de2-b094-7867363d00ce\") " pod="openshift-console/console-954f87865-4q2rj"
Oct 31 21:17:28.415430 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.415360    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a6becbb4-8451-4de2-b094-7867363d00ce-tmp\") pod \"console-954f87865-4q2rj\" (UID: \"a6becbb4-8451-4de2-b094-7867363d00ce\") " pod="openshift-console/console-954f87865-4q2rj"
Oct 31 21:17:28.415666 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.415472    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a6becbb4-8451-4de2-b094-7867363d00ce-console-serving-cert\") pod \"console-954f87865-4q2rj\" (UID: \"a6becbb4-8451-4de2-b094-7867363d00ce\") " pod="openshift-console/console-954f87865-4q2rj"
Oct 31 21:17:28.416027 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.415998    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a6becbb4-8451-4de2-b094-7867363d00ce-service-ca\") pod \"console-954f87865-4q2rj\" (UID: \"a6becbb4-8451-4de2-b094-7867363d00ce\") " pod="openshift-console/console-954f87865-4q2rj"
Oct 31 21:17:28.416198 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.416121    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a6becbb4-8451-4de2-b094-7867363d00ce-tmp\") pod \"console-954f87865-4q2rj\" (UID: \"a6becbb4-8451-4de2-b094-7867363d00ce\") " pod="openshift-console/console-954f87865-4q2rj"
Oct 31 21:17:28.416663 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.416628    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a6becbb4-8451-4de2-b094-7867363d00ce-console-config\") pod \"console-954f87865-4q2rj\" (UID: \"a6becbb4-8451-4de2-b094-7867363d00ce\") " pod="openshift-console/console-954f87865-4q2rj"
Oct 31 21:17:28.417075 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.417054    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a6becbb4-8451-4de2-b094-7867363d00ce-oauth-serving-cert\") pod \"console-954f87865-4q2rj\" (UID: \"a6becbb4-8451-4de2-b094-7867363d00ce\") " pod="openshift-console/console-954f87865-4q2rj"
Oct 31 21:17:28.417552 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.417481    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6becbb4-8451-4de2-b094-7867363d00ce-trusted-ca-bundle\") pod \"console-954f87865-4q2rj\" (UID: \"a6becbb4-8451-4de2-b094-7867363d00ce\") " pod="openshift-console/console-954f87865-4q2rj"
Oct 31 21:17:28.417552 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.417498    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:17:28.418228 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.418174    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a6becbb4-8451-4de2-b094-7867363d00ce-console-oauth-config\") pod \"console-954f87865-4q2rj\" (UID: \"a6becbb4-8451-4de2-b094-7867363d00ce\") " pod="openshift-console/console-954f87865-4q2rj"
Oct 31 21:17:28.418228 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.418193    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a6becbb4-8451-4de2-b094-7867363d00ce-console-serving-cert\") pod \"console-954f87865-4q2rj\" (UID: \"a6becbb4-8451-4de2-b094-7867363d00ce\") " pod="openshift-console/console-954f87865-4q2rj"
Oct 31 21:17:28.440348 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.440325    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwjtf\" (UniqueName: \"kubernetes.io/projected/a6becbb4-8451-4de2-b094-7867363d00ce-kube-api-access-qwjtf\") pod \"console-954f87865-4q2rj\" (UID: \"a6becbb4-8451-4de2-b094-7867363d00ce\") " pod="openshift-console/console-954f87865-4q2rj"
Oct 31 21:17:28.525662 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.525628    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:17:28.634186 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.634108    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-954f87865-4q2rj"
Oct 31 21:17:28.977040 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:28.976964    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-954f87865-4q2rj"]
Oct 31 21:17:28.979897 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:17:28.979872    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6becbb4_8451_4de2_b094_7867363d00ce.slice/crio-da3dd12a0e6bb92dda33f99bcf85f8973934a35047d08cbb7e49afb0dbe6853b WatchSource:0}: Error finding container da3dd12a0e6bb92dda33f99bcf85f8973934a35047d08cbb7e49afb0dbe6853b: Status 404 returned error can't find the container with id da3dd12a0e6bb92dda33f99bcf85f8973934a35047d08cbb7e49afb0dbe6853b
Oct 31 21:17:29.636647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:29.636604    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-954f87865-4q2rj" event={"ID":"a6becbb4-8451-4de2-b094-7867363d00ce","Type":"ContainerStarted","Data":"752e6d21ddf0dfc1bc8a94fcdc5c58bb1b6e7d3c731cac6d0a2f5a3958e5c5f0"}
Oct 31 21:17:29.637042 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:29.636656    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-954f87865-4q2rj" event={"ID":"a6becbb4-8451-4de2-b094-7867363d00ce","Type":"ContainerStarted","Data":"da3dd12a0e6bb92dda33f99bcf85f8973934a35047d08cbb7e49afb0dbe6853b"}
Oct 31 21:17:29.698791 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:29.698740    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-954f87865-4q2rj" podStartSLOduration=1.698723897 podStartE2EDuration="1.698723897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:17:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:17:29.684390753 +0000 UTC m=+89.600700184" watchObservedRunningTime="2026-10-31 21:17:29.698723897 +0000 UTC m=+89.615033327"
Oct 31 21:17:30.117469 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:30.117434    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:17:30.227438 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:30.227406    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:17:32.537757 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.537725    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 31 21:17:32.541847 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.541830    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.555226 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.555199    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.555401 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.555241    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.555401 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.555267    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.555401 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.555383    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-config-volume\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.555571 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.555417    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-config-out\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.555571 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.555492    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.555571 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.555525    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.555685 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.555579    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-web-config\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.555685 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.555610    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.555685 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.555635    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.555685 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.555660    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.555685 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.555681    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.555875 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.555696    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmj7h\" (UniqueName: \"kubernetes.io/projected/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-kube-api-access-cmj7h\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.558412 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.558388    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 31 21:17:32.571402 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.571378    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 31 21:17:32.571540 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.571409    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-c54qw\""
Oct 31 21:17:32.571540 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.571443    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 31 21:17:32.571540 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.571382    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 31 21:17:32.571540 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.571452    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 31 21:17:32.571540 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.571455    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 31 21:17:32.571540 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.571386    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 31 21:17:32.571540 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.571478    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 31 21:17:32.571868 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.571574    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 31 21:17:32.587149 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.587127    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 31 21:17:32.656054 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.656027    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.656200 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.656059    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-config-volume\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.656200 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.656077    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-config-out\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.656200 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.656094    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.656200 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.656110    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.656200 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.656144    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-web-config\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.656200 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.656167    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.656200 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.656181    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.656539 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.656206    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.656539 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.656238    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.656539 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.656260    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmj7h\" (UniqueName: \"kubernetes.io/projected/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-kube-api-access-cmj7h\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.656539 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.656323    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.656539 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.656343    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.656792 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.656735    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.657093 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.657028    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.657785 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.657763    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.659126 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.659093    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-config-volume\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.659236 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.659185    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.659354 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.659268    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.660087 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.660067    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.660177 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.660118    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-web-config\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.660177 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.660122    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.660177 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.660163    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-config-out\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.660312 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.660288    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.661065 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.661045    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.681635 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.681618    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmj7h\" (UniqueName: \"kubernetes.io/projected/2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b-kube-api-access-cmj7h\") pod \"alertmanager-main-0\" (UID: \"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:32.860887 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:32.860801    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 31 21:17:33.189255 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:33.189182    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 31 21:17:33.192305 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:17:33.192273    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fb86d0e_5f7e_48f4_b42c_d5a2096f5a2b.slice/crio-5b3be1482fcf7411aa41db49829d57b98d0ff8583c34f72e7f6e838c011f5798 WatchSource:0}: Error finding container 5b3be1482fcf7411aa41db49829d57b98d0ff8583c34f72e7f6e838c011f5798: Status 404 returned error can't find the container with id 5b3be1482fcf7411aa41db49829d57b98d0ff8583c34f72e7f6e838c011f5798
Oct 31 21:17:33.652785 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:33.652753    4460 generic.go:356] "Generic (PLEG): container finished" podID="2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b" containerID="282b4d955f648457eab7b0ec065ebc47fe7a3833bbd46b5147625b0df189ed51" exitCode=0
Oct 31 21:17:33.653281 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:33.652804    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b","Type":"ContainerDied","Data":"282b4d955f648457eab7b0ec065ebc47fe7a3833bbd46b5147625b0df189ed51"}
Oct 31 21:17:33.653281 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:33.652857    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b","Type":"ContainerStarted","Data":"5b3be1482fcf7411aa41db49829d57b98d0ff8583c34f72e7f6e838c011f5798"}
Oct 31 21:17:35.667576 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:35.667527    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b","Type":"ContainerStarted","Data":"99677accc5a8a64a18d97536ab030f69db7ed70edd19f7de0cc0b9f469f812d6"}
Oct 31 21:17:35.668110 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:35.667627    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b","Type":"ContainerStarted","Data":"fb86db22f9418a0fc0f7d111f70e2b345f8a8f57c96e35b7f9bc93abb557ad28"}
Oct 31 21:17:35.668110 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:35.667674    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b","Type":"ContainerStarted","Data":"74f52b6e3c415a9c7eb4ce2416ce9e1a81d0a27d2ee4cb75fda2bdaede10132b"}
Oct 31 21:17:35.668110 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:35.667715    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b","Type":"ContainerStarted","Data":"3a7336c5b42133043d019a2e0471f6439c1f1b6a76b28784afe4bde379d292cc"}
Oct 31 21:17:35.668110 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:35.667757    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b","Type":"ContainerStarted","Data":"a19a2b23c954fb944d92dbdad9acabb7cf0368dbeac44a9e12a30c9aec31419f"}
Oct 31 21:17:36.675848 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:36.675809    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b","Type":"ContainerStarted","Data":"6c8dba960a6c1731bd643100ea7a66046e85a8924d0031913f5c69f3e2de8cda"}
Oct 31 21:17:36.774591 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:36.774534    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.809096827 podStartE2EDuration="4.774517007s" totalImagesPullingTime="1.948819421s" totalInitContainerRuntime="16.600759ms" isStatefulPod=false podCreationTimestamp="2026-10-31 21:17:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:17:36.760336582 +0000 UTC m=+96.676646012" watchObservedRunningTime="2026-10-31 21:17:36.774517007 +0000 UTC m=+96.690826436"
Oct 31 21:17:38.220286 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:38.220247    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:17:38.330866 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:38.330817    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:17:38.634562 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:38.634519    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-954f87865-4q2rj"
Oct 31 21:17:38.634745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:38.634612    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-954f87865-4q2rj"
Oct 31 21:17:38.639163 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:38.639139    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-954f87865-4q2rj"
Oct 31 21:17:38.686469 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:38.686445    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-954f87865-4q2rj"
Oct 31 21:17:38.766907 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:38.766879    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7675b66b5c-9ffvv"]
Oct 31 21:17:43.002708 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:43.002665    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:17:51.451041 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:17:51.450997    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:17:53.770036 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:53.770000    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:17:53.926384 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:17:53.926346    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:18:03.785732 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:03.785680    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-7675b66b5c-9ffvv" podUID="e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b" containerName="console" containerID="cri-o://0de392c5f48a172c0f7f4f2ac6d8434671bb74f939a101de05dc2c82e3997a39" gracePeriod=15
Oct 31 21:18:04.015082 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.015059    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7675b66b5c-9ffvv"
Oct 31 21:18:04.101583 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.101509    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-service-ca\" (UniqueName: \"kubernetes.io/configmap/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-service-ca\") pod \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\" (UID: \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\") "
Oct 31 21:18:04.101733 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.101580    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-console-serving-cert\") pod \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\" (UID: \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\") "
Oct 31 21:18:04.101733 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.101606    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-kube-api-access-wb6q8\" (UniqueName: \"kubernetes.io/projected/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-kube-api-access-wb6q8\") pod \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\" (UID: \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\") "
Oct 31 21:18:04.101733 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.101629    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-console-oauth-config\") pod \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\" (UID: \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\") "
Oct 31 21:18:04.101733 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.101663    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-trusted-ca-bundle\") pod \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\" (UID: \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\") "
Oct 31 21:18:04.101733 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.101700    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-console-config\" (UniqueName: \"kubernetes.io/configmap/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-console-config\") pod \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\" (UID: \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\") "
Oct 31 21:18:04.102027 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.101747    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-tmp\" (UniqueName: \"kubernetes.io/empty-dir/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-tmp\") pod \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\" (UID: \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\") "
Oct 31 21:18:04.102027 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.101781    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-oauth-serving-cert\") pod \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\" (UID: \"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b\") "
Oct 31 21:18:04.102132 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.102098    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-service-ca" pod "e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b" (UID: "e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:18:04.102326 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.102293    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-trusted-ca-bundle" pod "e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b" (UID: "e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:18:04.102326 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.102311    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-tmp" pod "e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b" (UID: "e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:18:04.102451 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.102301    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-console-config" pod "e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b" (UID: "e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:18:04.102451 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.102348    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-oauth-serving-cert" pod "e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b" (UID: "e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:18:04.103852 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.103820    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-console-oauth-config" pod "e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b" (UID: "e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 21:18:04.103982 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.103867    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-console-serving-cert" pod "e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b" (UID: "e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 21:18:04.103982 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.103872    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-kube-api-access-wb6q8" pod "e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b" (UID: "e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b"). InnerVolumeSpecName "kube-api-access-wb6q8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:18:04.202678 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.202645    4460 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-oauth-serving-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:18:04.202678 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.202676    4460 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:18:04.202868 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.202690    4460 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-console-serving-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:18:04.202868 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.202703    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wb6q8\" (UniqueName: \"kubernetes.io/projected/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-kube-api-access-wb6q8\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:18:04.202868 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.202717    4460 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-console-oauth-config\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:18:04.202868 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.202729    4460 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-trusted-ca-bundle\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:18:04.202868 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.202741    4460 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-console-config\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:18:04.202868 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.202754    4460 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b-tmp\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:18:04.782817 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.782782    4460 generic.go:356] "Generic (PLEG): container finished" podID="e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b" containerID="0de392c5f48a172c0f7f4f2ac6d8434671bb74f939a101de05dc2c82e3997a39" exitCode=0
Oct 31 21:18:04.783004 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.782833    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7675b66b5c-9ffvv"
Oct 31 21:18:04.783004 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.782838    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7675b66b5c-9ffvv" event={"ID":"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b","Type":"ContainerDied","Data":"0de392c5f48a172c0f7f4f2ac6d8434671bb74f939a101de05dc2c82e3997a39"}
Oct 31 21:18:04.783004 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.782887    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7675b66b5c-9ffvv" event={"ID":"e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b","Type":"ContainerDied","Data":"5bce357e0ab5452702fd9b9811668b2d7187d26ea491c49364a658bbc28ed761"}
Oct 31 21:18:04.783004 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.782916    4460 scope.go:122] "RemoveContainer" containerID="0de392c5f48a172c0f7f4f2ac6d8434671bb74f939a101de05dc2c82e3997a39"
Oct 31 21:18:04.790768 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.790609    4460 scope.go:122] "RemoveContainer" containerID="0de392c5f48a172c0f7f4f2ac6d8434671bb74f939a101de05dc2c82e3997a39"
Oct 31 21:18:04.791003 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:18:04.790862    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0de392c5f48a172c0f7f4f2ac6d8434671bb74f939a101de05dc2c82e3997a39\": container with ID starting with 0de392c5f48a172c0f7f4f2ac6d8434671bb74f939a101de05dc2c82e3997a39 not found: ID does not exist" containerID="0de392c5f48a172c0f7f4f2ac6d8434671bb74f939a101de05dc2c82e3997a39"
Oct 31 21:18:04.791003 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.790888    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0de392c5f48a172c0f7f4f2ac6d8434671bb74f939a101de05dc2c82e3997a39"} err="failed to get container status \"0de392c5f48a172c0f7f4f2ac6d8434671bb74f939a101de05dc2c82e3997a39\": rpc error: code = NotFound desc = could not find container \"0de392c5f48a172c0f7f4f2ac6d8434671bb74f939a101de05dc2c82e3997a39\": container with ID starting with 0de392c5f48a172c0f7f4f2ac6d8434671bb74f939a101de05dc2c82e3997a39 not found: ID does not exist"
Oct 31 21:18:04.844117 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.844092    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7675b66b5c-9ffvv"]
Oct 31 21:18:04.857963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:04.857926    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7675b66b5c-9ffvv"]
Oct 31 21:18:06.221954 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:06.221913    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b" path="/var/lib/kubelet/pods/e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b/volumes"
Oct 31 21:18:21.450457 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:18:21.450372    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:18:22.215473 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:22.215438    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:18:22.329532 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:22.329501    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:18:36.723112 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:36.723073    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:18:36.843802 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:36.843763    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:18:43.117135 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:43.117096    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:18:43.228731 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:18:43.228692    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:18:51.451282 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:18:51.451249    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:19:08.637466 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:19:08.637427    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:19:08.765536 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:19:08.765503    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:19:14.663578 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:19:14.663534    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:19:21.451278 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:19:21.451233    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:19:43.235901 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:19:43.235810    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:19:51.450840 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:19:51.450801    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:19:55.389272 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:19:55.389239    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:20:09.107962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:09.107906    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:20:21.450462 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:20:21.450424    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:20:21.534080 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:21.534047    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:20:21.914302 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:21.914269    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:20:26.308356 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:26.308310    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:20:26.463009 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:26.462974    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:20:26.587903 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:26.587805    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:20:27.444622 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:27.444592    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4nz6"]
Oct 31 21:20:27.445042 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:27.444893    4460 cpu_manager.go:414] "containerMap: removing container" podUID="e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b" containerName="console"
Oct 31 21:20:27.445042 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:27.444904    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b" containerName="console"
Oct 31 21:20:27.445042 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:27.444985    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="e00a562f-5d3f-4c5e-9be5-4d0bb4197f5b" containerName="console"
Oct 31 21:20:27.448020 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:27.448003    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4nz6"
Oct 31 21:20:27.467828 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:27.467805    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 31 21:20:27.467955 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:27.467838    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-gkgc4\""
Oct 31 21:20:27.481434 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:27.481411    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 31 21:20:27.500268 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:27.500236    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4nz6"]
Oct 31 21:20:27.554751 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:27.554721    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9e03596-7703-4524-b03e-b7e3f5a03977-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4nz6\" (UID: \"a9e03596-7703-4524-b03e-b7e3f5a03977\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4nz6"
Oct 31 21:20:27.554893 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:27.554763    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwpnf\" (UniqueName: \"kubernetes.io/projected/a9e03596-7703-4524-b03e-b7e3f5a03977-kube-api-access-dwpnf\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4nz6\" (UID: \"a9e03596-7703-4524-b03e-b7e3f5a03977\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4nz6"
Oct 31 21:20:27.554893 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:27.554831    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9e03596-7703-4524-b03e-b7e3f5a03977-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4nz6\" (UID: \"a9e03596-7703-4524-b03e-b7e3f5a03977\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4nz6"
Oct 31 21:20:27.614735 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:27.614694    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ksmt8"]
Oct 31 21:20:27.618144 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:27.618121    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ksmt8"
Oct 31 21:20:27.635682 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:27.635652    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-9hcw6\""
Oct 31 21:20:27.639711 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:27.639684    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ksmt8"]
Oct 31 21:20:27.656088 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:27.656055    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9e03596-7703-4524-b03e-b7e3f5a03977-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4nz6\" (UID: \"a9e03596-7703-4524-b03e-b7e3f5a03977\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4nz6"
Oct 31 21:20:27.656218 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:27.656098    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwpnf\" (UniqueName: \"kubernetes.io/projected/a9e03596-7703-4524-b03e-b7e3f5a03977-kube-api-access-dwpnf\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4nz6\" (UID: \"a9e03596-7703-4524-b03e-b7e3f5a03977\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4nz6"
Oct 31 21:20:27.656218 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:27.656120    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9e03596-7703-4524-b03e-b7e3f5a03977-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4nz6\" (UID: \"a9e03596-7703-4524-b03e-b7e3f5a03977\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4nz6"
Oct 31 21:20:27.656448 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:27.656433    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9e03596-7703-4524-b03e-b7e3f5a03977-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4nz6\" (UID: \"a9e03596-7703-4524-b03e-b7e3f5a03977\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4nz6"
Oct 31 21:20:27.656528 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:27.656511    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9e03596-7703-4524-b03e-b7e3f5a03977-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4nz6\" (UID: \"a9e03596-7703-4524-b03e-b7e3f5a03977\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4nz6"
Oct 31 21:20:27.681537 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:27.681505    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwpnf\" (UniqueName: \"kubernetes.io/projected/a9e03596-7703-4524-b03e-b7e3f5a03977-kube-api-access-dwpnf\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4nz6\" (UID: \"a9e03596-7703-4524-b03e-b7e3f5a03977\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4nz6"
Oct 31 21:20:27.756584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:27.756491    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9502d2bb-0ec4-410d-9c07-807f33a10cf7-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ksmt8\" (UID: \"9502d2bb-0ec4-410d-9c07-807f33a10cf7\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ksmt8"
Oct 31 21:20:27.756584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:27.756535    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxshk\" (UniqueName: \"kubernetes.io/projected/9502d2bb-0ec4-410d-9c07-807f33a10cf7-kube-api-access-jxshk\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ksmt8\" (UID: \"9502d2bb-0ec4-410d-9c07-807f33a10cf7\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ksmt8"
Oct 31 21:20:27.756773 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:27.756645    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9502d2bb-0ec4-410d-9c07-807f33a10cf7-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ksmt8\" (UID: \"9502d2bb-0ec4-410d-9c07-807f33a10cf7\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ksmt8"
Oct 31 21:20:27.764443 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:27.764411    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4nz6"
Oct 31 21:20:27.857466 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:27.857427    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9502d2bb-0ec4-410d-9c07-807f33a10cf7-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ksmt8\" (UID: \"9502d2bb-0ec4-410d-9c07-807f33a10cf7\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ksmt8"
Oct 31 21:20:27.857626 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:27.857493    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9502d2bb-0ec4-410d-9c07-807f33a10cf7-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ksmt8\" (UID: \"9502d2bb-0ec4-410d-9c07-807f33a10cf7\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ksmt8"
Oct 31 21:20:27.857626 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:27.857523    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxshk\" (UniqueName: \"kubernetes.io/projected/9502d2bb-0ec4-410d-9c07-807f33a10cf7-kube-api-access-jxshk\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ksmt8\" (UID: \"9502d2bb-0ec4-410d-9c07-807f33a10cf7\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ksmt8"
Oct 31 21:20:27.857926 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:27.857887    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9502d2bb-0ec4-410d-9c07-807f33a10cf7-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ksmt8\" (UID: \"9502d2bb-0ec4-410d-9c07-807f33a10cf7\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ksmt8"
Oct 31 21:20:27.857993 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:27.857908    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9502d2bb-0ec4-410d-9c07-807f33a10cf7-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ksmt8\" (UID: \"9502d2bb-0ec4-410d-9c07-807f33a10cf7\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ksmt8"
Oct 31 21:20:27.883396 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:27.883362    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxshk\" (UniqueName: \"kubernetes.io/projected/9502d2bb-0ec4-410d-9c07-807f33a10cf7-kube-api-access-jxshk\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ksmt8\" (UID: \"9502d2bb-0ec4-410d-9c07-807f33a10cf7\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ksmt8"
Oct 31 21:20:27.937506 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:27.937472    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ksmt8"
Oct 31 21:20:28.054775 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.054745    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf"]
Oct 31 21:20:28.058208 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.058185    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf"
Oct 31 21:20:28.064661 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.064638    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4nz6"]
Oct 31 21:20:28.068049 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:20:28.068024    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9e03596_7703_4524_b03e_b7e3f5a03977.slice/crio-c41ae3d2173d930ba79bd7319cd5e8410fbe5fed062d949d6a6e07f0b211b8bb WatchSource:0}: Error finding container c41ae3d2173d930ba79bd7319cd5e8410fbe5fed062d949d6a6e07f0b211b8bb: Status 404 returned error can't find the container with id c41ae3d2173d930ba79bd7319cd5e8410fbe5fed062d949d6a6e07f0b211b8bb
Oct 31 21:20:28.073244 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.073214    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:20:28.074500 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.074482    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 31 21:20:28.074579 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.074483    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 31 21:20:28.074854 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.074840    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 31 21:20:28.075227 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.075212    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 31 21:20:28.075390 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.075374    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 31 21:20:28.075453 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.075379    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 31 21:20:28.075507 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.075494    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-wxtz5\""
Oct 31 21:20:28.085739 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.085708    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf"]
Oct 31 21:20:28.160415 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.160375    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a70a6be8-a62b-4159-bff1-8837e3791e03-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-jkfxf\" (UID: \"a70a6be8-a62b-4159-bff1-8837e3791e03\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf"
Oct 31 21:20:28.160596 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.160423    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a70a6be8-a62b-4159-bff1-8837e3791e03-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-jkfxf\" (UID: \"a70a6be8-a62b-4159-bff1-8837e3791e03\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf"
Oct 31 21:20:28.160596 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.160493    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a70a6be8-a62b-4159-bff1-8837e3791e03-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-jkfxf\" (UID: \"a70a6be8-a62b-4159-bff1-8837e3791e03\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf"
Oct 31 21:20:28.160596 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.160517    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a70a6be8-a62b-4159-bff1-8837e3791e03-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-jkfxf\" (UID: \"a70a6be8-a62b-4159-bff1-8837e3791e03\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf"
Oct 31 21:20:28.160596 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.160547    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a70a6be8-a62b-4159-bff1-8837e3791e03-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-jkfxf\" (UID: \"a70a6be8-a62b-4159-bff1-8837e3791e03\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf"
Oct 31 21:20:28.160596 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.160564    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a70a6be8-a62b-4159-bff1-8837e3791e03-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-jkfxf\" (UID: \"a70a6be8-a62b-4159-bff1-8837e3791e03\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf"
Oct 31 21:20:28.160753 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.160623    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnl48\" (UniqueName: \"kubernetes.io/projected/a70a6be8-a62b-4159-bff1-8837e3791e03-kube-api-access-dnl48\") pod \"istiod-openshift-gateway-7876bfcb74-jkfxf\" (UID: \"a70a6be8-a62b-4159-bff1-8837e3791e03\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf"
Oct 31 21:20:28.189097 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.189058    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:20:28.235403 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.235375    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ksmt8"]
Oct 31 21:20:28.238347 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:20:28.238317    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9502d2bb_0ec4_410d_9c07_807f33a10cf7.slice/crio-76c8365c651a4e1e69482a02e8780648e4290149f65672b89a9e3765a084d1d3 WatchSource:0}: Error finding container 76c8365c651a4e1e69482a02e8780648e4290149f65672b89a9e3765a084d1d3: Status 404 returned error can't find the container with id 76c8365c651a4e1e69482a02e8780648e4290149f65672b89a9e3765a084d1d3
Oct 31 21:20:28.261635 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.261601    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a70a6be8-a62b-4159-bff1-8837e3791e03-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-jkfxf\" (UID: \"a70a6be8-a62b-4159-bff1-8837e3791e03\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf"
Oct 31 21:20:28.261635 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.261639    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a70a6be8-a62b-4159-bff1-8837e3791e03-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-jkfxf\" (UID: \"a70a6be8-a62b-4159-bff1-8837e3791e03\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf"
Oct 31 21:20:28.261797 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.261655    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a70a6be8-a62b-4159-bff1-8837e3791e03-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-jkfxf\" (UID: \"a70a6be8-a62b-4159-bff1-8837e3791e03\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf"
Oct 31 21:20:28.261797 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.261674    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnl48\" (UniqueName: \"kubernetes.io/projected/a70a6be8-a62b-4159-bff1-8837e3791e03-kube-api-access-dnl48\") pod \"istiod-openshift-gateway-7876bfcb74-jkfxf\" (UID: \"a70a6be8-a62b-4159-bff1-8837e3791e03\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf"
Oct 31 21:20:28.261872 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.261791    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a70a6be8-a62b-4159-bff1-8837e3791e03-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-jkfxf\" (UID: \"a70a6be8-a62b-4159-bff1-8837e3791e03\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf"
Oct 31 21:20:28.261872 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.261844    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a70a6be8-a62b-4159-bff1-8837e3791e03-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-jkfxf\" (UID: \"a70a6be8-a62b-4159-bff1-8837e3791e03\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf"
Oct 31 21:20:28.261965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.261885    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a70a6be8-a62b-4159-bff1-8837e3791e03-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-jkfxf\" (UID: \"a70a6be8-a62b-4159-bff1-8837e3791e03\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf"
Oct 31 21:20:28.262488 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.262464    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a70a6be8-a62b-4159-bff1-8837e3791e03-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-jkfxf\" (UID: \"a70a6be8-a62b-4159-bff1-8837e3791e03\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf"
Oct 31 21:20:28.263968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.263934    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a70a6be8-a62b-4159-bff1-8837e3791e03-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-jkfxf\" (UID: \"a70a6be8-a62b-4159-bff1-8837e3791e03\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf"
Oct 31 21:20:28.264109 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.264085    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a70a6be8-a62b-4159-bff1-8837e3791e03-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-jkfxf\" (UID: \"a70a6be8-a62b-4159-bff1-8837e3791e03\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf"
Oct 31 21:20:28.264343 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.264329    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a70a6be8-a62b-4159-bff1-8837e3791e03-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-jkfxf\" (UID: \"a70a6be8-a62b-4159-bff1-8837e3791e03\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf"
Oct 31 21:20:28.264380 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.264370    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a70a6be8-a62b-4159-bff1-8837e3791e03-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-jkfxf\" (UID: \"a70a6be8-a62b-4159-bff1-8837e3791e03\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf"
Oct 31 21:20:28.274120 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.274090    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4nz6" event={"ID":"a9e03596-7703-4524-b03e-b7e3f5a03977","Type":"ContainerStarted","Data":"c41ae3d2173d930ba79bd7319cd5e8410fbe5fed062d949d6a6e07f0b211b8bb"}
Oct 31 21:20:28.275132 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.275111    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ksmt8" event={"ID":"9502d2bb-0ec4-410d-9c07-807f33a10cf7","Type":"ContainerStarted","Data":"76c8365c651a4e1e69482a02e8780648e4290149f65672b89a9e3765a084d1d3"}
Oct 31 21:20:28.290435 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.290404    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a70a6be8-a62b-4159-bff1-8837e3791e03-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-jkfxf\" (UID: \"a70a6be8-a62b-4159-bff1-8837e3791e03\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf"
Oct 31 21:20:28.290622 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.290602    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnl48\" (UniqueName: \"kubernetes.io/projected/a70a6be8-a62b-4159-bff1-8837e3791e03-kube-api-access-dnl48\") pod \"istiod-openshift-gateway-7876bfcb74-jkfxf\" (UID: \"a70a6be8-a62b-4159-bff1-8837e3791e03\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf"
Oct 31 21:20:28.372094 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.372011    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf"
Oct 31 21:20:28.683715 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:28.683630    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf"]
Oct 31 21:20:28.688176 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:20:28.688144    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda70a6be8_a62b_4159_bff1_8837e3791e03.slice/crio-df46fcacb035d97faaa6be90710a536c28ab68c05d9212d7a427e812b69b23d4 WatchSource:0}: Error finding container df46fcacb035d97faaa6be90710a536c28ab68c05d9212d7a427e812b69b23d4: Status 404 returned error can't find the container with id df46fcacb035d97faaa6be90710a536c28ab68c05d9212d7a427e812b69b23d4
Oct 31 21:20:29.285448 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:29.285407    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf" event={"ID":"a70a6be8-a62b-4159-bff1-8837e3791e03","Type":"ContainerStarted","Data":"df46fcacb035d97faaa6be90710a536c28ab68c05d9212d7a427e812b69b23d4"}
Oct 31 21:20:29.568834 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:29.568748    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:20:31.815453 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:31.815420    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:20:31.815734 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:31.815523    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:20:32.305969 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:32.303633    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf" event={"ID":"a70a6be8-a62b-4159-bff1-8837e3791e03","Type":"ContainerStarted","Data":"02f50ecd11292f9fcbfa47b3604435087a01ea769083b6f7302e08c4ad4fbe4f"}
Oct 31 21:20:32.305969 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:32.303820    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf"
Oct 31 21:20:32.305969 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:32.305922    4460 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-jkfxf container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 21:20:32.306218 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:32.305987    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf" podUID="a70a6be8-a62b-4159-bff1-8837e3791e03" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 21:20:32.368619 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:32.368558    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf" podStartSLOduration=1.244543611 podStartE2EDuration="4.368537489s" totalImagesPullingTime="3.123993878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:20:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:20:32.366991073 +0000 UTC m=+272.283300499" watchObservedRunningTime="2026-10-31 21:20:32.368537489 +0000 UTC m=+272.284846920"
Oct 31 21:20:32.549098 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:32.549053    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:20:32.682535 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:32.682439    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:20:33.308529 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:33.308489    4460 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-jkfxf container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 21:20:33.308999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:33.308562    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf" podUID="a70a6be8-a62b-4159-bff1-8837e3791e03" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 21:20:34.108022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:34.107983    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:20:35.317070 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:35.316978    4460 generic.go:356] "Generic (PLEG): container finished" podID="a9e03596-7703-4524-b03e-b7e3f5a03977" containerID="0fbfa46568790bdce11639a9ef02d1566c4570744e6ccd59ad2a357e92fb114f" exitCode=0
Oct 31 21:20:35.317450 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:35.317062    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4nz6" event={"ID":"a9e03596-7703-4524-b03e-b7e3f5a03977","Type":"ContainerDied","Data":"0fbfa46568790bdce11639a9ef02d1566c4570744e6ccd59ad2a357e92fb114f"}
Oct 31 21:20:35.318722 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:35.318702    4460 generic.go:356] "Generic (PLEG): container finished" podID="9502d2bb-0ec4-410d-9c07-807f33a10cf7" containerID="5e798e6501d93abff13d7599d87a25746f8d70498c72353a61d29935a33ee5a4" exitCode=0
Oct 31 21:20:35.318831 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:35.318761    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ksmt8" event={"ID":"9502d2bb-0ec4-410d-9c07-807f33a10cf7","Type":"ContainerDied","Data":"5e798e6501d93abff13d7599d87a25746f8d70498c72353a61d29935a33ee5a4"}
Oct 31 21:20:35.429093 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:35.429051    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:20:36.309232 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:36.309199    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf"
Oct 31 21:20:36.325379 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:36.325348    4460 generic.go:356] "Generic (PLEG): container finished" podID="9502d2bb-0ec4-410d-9c07-807f33a10cf7" containerID="8f06232463a4fe7750684acc35d91086b1c7cff02e2ae61f32a7c17cf524591c" exitCode=0
Oct 31 21:20:36.325797 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:36.325387    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ksmt8" event={"ID":"9502d2bb-0ec4-410d-9c07-807f33a10cf7","Type":"ContainerDied","Data":"8f06232463a4fe7750684acc35d91086b1c7cff02e2ae61f32a7c17cf524591c"}
Oct 31 21:20:37.453350 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:37.453314    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:20:38.337960 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:38.337902    4460 generic.go:356] "Generic (PLEG): container finished" podID="a9e03596-7703-4524-b03e-b7e3f5a03977" containerID="14f8d49b7fc14d6abfe29b847852719f29feeac935b04bb7319df22fc1a3ae1b" exitCode=0
Oct 31 21:20:38.338117 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:38.337965    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4nz6" event={"ID":"a9e03596-7703-4524-b03e-b7e3f5a03977","Type":"ContainerDied","Data":"14f8d49b7fc14d6abfe29b847852719f29feeac935b04bb7319df22fc1a3ae1b"}
Oct 31 21:20:43.365859 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:43.365819    4460 generic.go:356] "Generic (PLEG): container finished" podID="9502d2bb-0ec4-410d-9c07-807f33a10cf7" containerID="e5943bf48e26ea38d2fdd3b61cbd4515c7bdd33ed63a7dce1a4d7b15ba45ca6c" exitCode=0
Oct 31 21:20:43.366334 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:43.365898    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ksmt8" event={"ID":"9502d2bb-0ec4-410d-9c07-807f33a10cf7","Type":"ContainerDied","Data":"e5943bf48e26ea38d2fdd3b61cbd4515c7bdd33ed63a7dce1a4d7b15ba45ca6c"}
Oct 31 21:20:43.378891 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:43.378860    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:20:44.488176 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:44.488155    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ksmt8"
Oct 31 21:20:44.520304 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:44.520279    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9502d2bb-0ec4-410d-9c07-807f33a10cf7-bundle\" (UniqueName: \"kubernetes.io/empty-dir/9502d2bb-0ec4-410d-9c07-807f33a10cf7-bundle\") pod \"9502d2bb-0ec4-410d-9c07-807f33a10cf7\" (UID: \"9502d2bb-0ec4-410d-9c07-807f33a10cf7\") "
Oct 31 21:20:44.520430 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:44.520324    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9502d2bb-0ec4-410d-9c07-807f33a10cf7-kube-api-access-jxshk\" (UniqueName: \"kubernetes.io/projected/9502d2bb-0ec4-410d-9c07-807f33a10cf7-kube-api-access-jxshk\") pod \"9502d2bb-0ec4-410d-9c07-807f33a10cf7\" (UID: \"9502d2bb-0ec4-410d-9c07-807f33a10cf7\") "
Oct 31 21:20:44.520493 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:44.520427    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9502d2bb-0ec4-410d-9c07-807f33a10cf7-util\" (UniqueName: \"kubernetes.io/empty-dir/9502d2bb-0ec4-410d-9c07-807f33a10cf7-util\") pod \"9502d2bb-0ec4-410d-9c07-807f33a10cf7\" (UID: \"9502d2bb-0ec4-410d-9c07-807f33a10cf7\") "
Oct 31 21:20:44.520903 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:44.520879    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9502d2bb-0ec4-410d-9c07-807f33a10cf7-bundle" pod "9502d2bb-0ec4-410d-9c07-807f33a10cf7" (UID: "9502d2bb-0ec4-410d-9c07-807f33a10cf7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:20:44.522669 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:44.522648    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9502d2bb-0ec4-410d-9c07-807f33a10cf7-kube-api-access-jxshk" pod "9502d2bb-0ec4-410d-9c07-807f33a10cf7" (UID: "9502d2bb-0ec4-410d-9c07-807f33a10cf7"). InnerVolumeSpecName "kube-api-access-jxshk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:20:44.526097 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:44.526065    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9502d2bb-0ec4-410d-9c07-807f33a10cf7-util" pod "9502d2bb-0ec4-410d-9c07-807f33a10cf7" (UID: "9502d2bb-0ec4-410d-9c07-807f33a10cf7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:20:44.621099 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:44.621028    4460 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9502d2bb-0ec4-410d-9c07-807f33a10cf7-bundle\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:20:44.621099 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:44.621055    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxshk\" (UniqueName: \"kubernetes.io/projected/9502d2bb-0ec4-410d-9c07-807f33a10cf7-kube-api-access-jxshk\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:20:44.621099 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:44.621065    4460 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9502d2bb-0ec4-410d-9c07-807f33a10cf7-util\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:20:45.375379 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:45.375346    4460 generic.go:356] "Generic (PLEG): container finished" podID="a9e03596-7703-4524-b03e-b7e3f5a03977" containerID="bb380d2b6e9af58a457f576ea573a37169375c544a455ba09dc4a6eaadc83d7a" exitCode=0
Oct 31 21:20:45.375549 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:45.375439    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4nz6" event={"ID":"a9e03596-7703-4524-b03e-b7e3f5a03977","Type":"ContainerDied","Data":"bb380d2b6e9af58a457f576ea573a37169375c544a455ba09dc4a6eaadc83d7a"}
Oct 31 21:20:45.377263 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:45.377237    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ksmt8" event={"ID":"9502d2bb-0ec4-410d-9c07-807f33a10cf7","Type":"ContainerDied","Data":"76c8365c651a4e1e69482a02e8780648e4290149f65672b89a9e3765a084d1d3"}
Oct 31 21:20:45.377263 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:45.377259    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71ksmt8"
Oct 31 21:20:45.377424 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:45.377278    4460 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="76c8365c651a4e1e69482a02e8780648e4290149f65672b89a9e3765a084d1d3"
Oct 31 21:20:46.500394 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:46.500371    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4nz6"
Oct 31 21:20:46.534321 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:46.534293    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9e03596-7703-4524-b03e-b7e3f5a03977-util\" (UniqueName: \"kubernetes.io/empty-dir/a9e03596-7703-4524-b03e-b7e3f5a03977-util\") pod \"a9e03596-7703-4524-b03e-b7e3f5a03977\" (UID: \"a9e03596-7703-4524-b03e-b7e3f5a03977\") "
Oct 31 21:20:46.534431 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:46.534328    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9e03596-7703-4524-b03e-b7e3f5a03977-bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9e03596-7703-4524-b03e-b7e3f5a03977-bundle\") pod \"a9e03596-7703-4524-b03e-b7e3f5a03977\" (UID: \"a9e03596-7703-4524-b03e-b7e3f5a03977\") "
Oct 31 21:20:46.534431 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:46.534377    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9e03596-7703-4524-b03e-b7e3f5a03977-kube-api-access-dwpnf\" (UniqueName: \"kubernetes.io/projected/a9e03596-7703-4524-b03e-b7e3f5a03977-kube-api-access-dwpnf\") pod \"a9e03596-7703-4524-b03e-b7e3f5a03977\" (UID: \"a9e03596-7703-4524-b03e-b7e3f5a03977\") "
Oct 31 21:20:46.534760 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:46.534734    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9e03596-7703-4524-b03e-b7e3f5a03977-bundle" pod "a9e03596-7703-4524-b03e-b7e3f5a03977" (UID: "a9e03596-7703-4524-b03e-b7e3f5a03977"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:20:46.536420 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:46.536394    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9e03596-7703-4524-b03e-b7e3f5a03977-kube-api-access-dwpnf" pod "a9e03596-7703-4524-b03e-b7e3f5a03977" (UID: "a9e03596-7703-4524-b03e-b7e3f5a03977"). InnerVolumeSpecName "kube-api-access-dwpnf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:20:46.538048 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:46.538028    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9e03596-7703-4524-b03e-b7e3f5a03977-util" pod "a9e03596-7703-4524-b03e-b7e3f5a03977" (UID: "a9e03596-7703-4524-b03e-b7e3f5a03977"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:20:46.635325 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:46.635253    4460 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a9e03596-7703-4524-b03e-b7e3f5a03977-util\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:20:46.635325 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:46.635274    4460 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a9e03596-7703-4524-b03e-b7e3f5a03977-bundle\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:20:46.635325 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:46.635284    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwpnf\" (UniqueName: \"kubernetes.io/projected/a9e03596-7703-4524-b03e-b7e3f5a03977-kube-api-access-dwpnf\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:20:47.385402 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:47.385371    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4nz6"
Oct 31 21:20:47.385402 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:47.385392    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644s4nz6" event={"ID":"a9e03596-7703-4524-b03e-b7e3f5a03977","Type":"ContainerDied","Data":"c41ae3d2173d930ba79bd7319cd5e8410fbe5fed062d949d6a6e07f0b211b8bb"}
Oct 31 21:20:47.385598 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:20:47.385440    4460 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c41ae3d2173d930ba79bd7319cd5e8410fbe5fed062d949d6a6e07f0b211b8bb"
Oct 31 21:20:51.450329 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:20:51.450292    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:21:00.230915 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:00.230889    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ssl9_26069598-5dc4-4c96-b55f-dd2f09884419/ovn-acl-logging/0.log"
Oct 31 21:21:00.230915 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:00.230911    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ssl9_26069598-5dc4-4c96-b55f-dd2f09884419/ovn-acl-logging/0.log"
Oct 31 21:21:03.642441 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:03.642402    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:03.761916 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:03.761871    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:10.563364 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:10.563325    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:10.704107 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:10.704074    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:11.855579 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:11.855542    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:12.076311 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.076277    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-899c649cb-gs59t"]
Oct 31 21:21:12.076589 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.076577    4460 cpu_manager.go:414] "containerMap: removing container" podUID="9502d2bb-0ec4-410d-9c07-807f33a10cf7" containerName="util"
Oct 31 21:21:12.076633 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.076590    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9502d2bb-0ec4-410d-9c07-807f33a10cf7" containerName="util"
Oct 31 21:21:12.076633 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.076606    4460 cpu_manager.go:414] "containerMap: removing container" podUID="9502d2bb-0ec4-410d-9c07-807f33a10cf7" containerName="extract"
Oct 31 21:21:12.076633 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.076610    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9502d2bb-0ec4-410d-9c07-807f33a10cf7" containerName="extract"
Oct 31 21:21:12.076633 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.076617    4460 cpu_manager.go:414] "containerMap: removing container" podUID="a9e03596-7703-4524-b03e-b7e3f5a03977" containerName="extract"
Oct 31 21:21:12.076633 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.076622    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9e03596-7703-4524-b03e-b7e3f5a03977" containerName="extract"
Oct 31 21:21:12.076633 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.076632    4460 cpu_manager.go:414] "containerMap: removing container" podUID="a9e03596-7703-4524-b03e-b7e3f5a03977" containerName="pull"
Oct 31 21:21:12.076804 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.076636    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9e03596-7703-4524-b03e-b7e3f5a03977" containerName="pull"
Oct 31 21:21:12.076804 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.076659    4460 cpu_manager.go:414] "containerMap: removing container" podUID="a9e03596-7703-4524-b03e-b7e3f5a03977" containerName="util"
Oct 31 21:21:12.076804 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.076664    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9e03596-7703-4524-b03e-b7e3f5a03977" containerName="util"
Oct 31 21:21:12.076804 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.076669    4460 cpu_manager.go:414] "containerMap: removing container" podUID="9502d2bb-0ec4-410d-9c07-807f33a10cf7" containerName="pull"
Oct 31 21:21:12.076804 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.076674    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9502d2bb-0ec4-410d-9c07-807f33a10cf7" containerName="pull"
Oct 31 21:21:12.076804 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.076717    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="9502d2bb-0ec4-410d-9c07-807f33a10cf7" containerName="extract"
Oct 31 21:21:12.076804 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.076724    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9e03596-7703-4524-b03e-b7e3f5a03977" containerName="extract"
Oct 31 21:21:12.080475 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.080457    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-899c649cb-gs59t"
Oct 31 21:21:12.090457 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.090435    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-899c649cb-gs59t"]
Oct 31 21:21:12.115834 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.115766    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znj4h\" (UniqueName: \"kubernetes.io/projected/783fffe8-eda8-419f-ac06-320afd1ab891-kube-api-access-znj4h\") pod \"opendatahub-operator-controller-manager-899c649cb-gs59t\" (UID: \"783fffe8-eda8-419f-ac06-320afd1ab891\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-899c649cb-gs59t"
Oct 31 21:21:12.115834 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.115805    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/783fffe8-eda8-419f-ac06-320afd1ab891-webhook-cert\") pod \"opendatahub-operator-controller-manager-899c649cb-gs59t\" (UID: \"783fffe8-eda8-419f-ac06-320afd1ab891\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-899c649cb-gs59t"
Oct 31 21:21:12.116012 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.115895    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/783fffe8-eda8-419f-ac06-320afd1ab891-metrics-cert\") pod \"opendatahub-operator-controller-manager-899c649cb-gs59t\" (UID: \"783fffe8-eda8-419f-ac06-320afd1ab891\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-899c649cb-gs59t"
Oct 31 21:21:12.116012 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.115991    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/783fffe8-eda8-419f-ac06-320afd1ab891-tmp\") pod \"opendatahub-operator-controller-manager-899c649cb-gs59t\" (UID: \"783fffe8-eda8-419f-ac06-320afd1ab891\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-899c649cb-gs59t"
Oct 31 21:21:12.116076 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.116037    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/783fffe8-eda8-419f-ac06-320afd1ab891-manifests\") pod \"opendatahub-operator-controller-manager-899c649cb-gs59t\" (UID: \"783fffe8-eda8-419f-ac06-320afd1ab891\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-899c649cb-gs59t"
Oct 31 21:21:12.116076 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.116058    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/783fffe8-eda8-419f-ac06-320afd1ab891-apiservice-cert\") pod \"opendatahub-operator-controller-manager-899c649cb-gs59t\" (UID: \"783fffe8-eda8-419f-ac06-320afd1ab891\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-899c649cb-gs59t"
Oct 31 21:21:12.167656 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.167629    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Oct 31 21:21:12.167799 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.167709    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 31 21:21:12.167799 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.167758    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Oct 31 21:21:12.167799 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.167795    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 31 21:21:12.167907 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.167864    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-lmxhd\""
Oct 31 21:21:12.167971 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.167958    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Oct 31 21:21:12.194234 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.194207    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:12.217438 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.217409    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znj4h\" (UniqueName: \"kubernetes.io/projected/783fffe8-eda8-419f-ac06-320afd1ab891-kube-api-access-znj4h\") pod \"opendatahub-operator-controller-manager-899c649cb-gs59t\" (UID: \"783fffe8-eda8-419f-ac06-320afd1ab891\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-899c649cb-gs59t"
Oct 31 21:21:12.217549 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.217446    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/783fffe8-eda8-419f-ac06-320afd1ab891-webhook-cert\") pod \"opendatahub-operator-controller-manager-899c649cb-gs59t\" (UID: \"783fffe8-eda8-419f-ac06-320afd1ab891\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-899c649cb-gs59t"
Oct 31 21:21:12.217549 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.217487    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/783fffe8-eda8-419f-ac06-320afd1ab891-metrics-cert\") pod \"opendatahub-operator-controller-manager-899c649cb-gs59t\" (UID: \"783fffe8-eda8-419f-ac06-320afd1ab891\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-899c649cb-gs59t"
Oct 31 21:21:12.217549 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.217523    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/783fffe8-eda8-419f-ac06-320afd1ab891-tmp\") pod \"opendatahub-operator-controller-manager-899c649cb-gs59t\" (UID: \"783fffe8-eda8-419f-ac06-320afd1ab891\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-899c649cb-gs59t"
Oct 31 21:21:12.217672 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.217566    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/783fffe8-eda8-419f-ac06-320afd1ab891-manifests\") pod \"opendatahub-operator-controller-manager-899c649cb-gs59t\" (UID: \"783fffe8-eda8-419f-ac06-320afd1ab891\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-899c649cb-gs59t"
Oct 31 21:21:12.217672 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.217588    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/783fffe8-eda8-419f-ac06-320afd1ab891-apiservice-cert\") pod \"opendatahub-operator-controller-manager-899c649cb-gs59t\" (UID: \"783fffe8-eda8-419f-ac06-320afd1ab891\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-899c649cb-gs59t"
Oct 31 21:21:12.218034 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.218014    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/783fffe8-eda8-419f-ac06-320afd1ab891-manifests\") pod \"opendatahub-operator-controller-manager-899c649cb-gs59t\" (UID: \"783fffe8-eda8-419f-ac06-320afd1ab891\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-899c649cb-gs59t"
Oct 31 21:21:12.220157 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.220134    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/783fffe8-eda8-419f-ac06-320afd1ab891-tmp\") pod \"opendatahub-operator-controller-manager-899c649cb-gs59t\" (UID: \"783fffe8-eda8-419f-ac06-320afd1ab891\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-899c649cb-gs59t"
Oct 31 21:21:12.220248 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.220230    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/783fffe8-eda8-419f-ac06-320afd1ab891-apiservice-cert\") pod \"opendatahub-operator-controller-manager-899c649cb-gs59t\" (UID: \"783fffe8-eda8-419f-ac06-320afd1ab891\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-899c649cb-gs59t"
Oct 31 21:21:12.220314 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.220292    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/783fffe8-eda8-419f-ac06-320afd1ab891-webhook-cert\") pod \"opendatahub-operator-controller-manager-899c649cb-gs59t\" (UID: \"783fffe8-eda8-419f-ac06-320afd1ab891\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-899c649cb-gs59t"
Oct 31 21:21:12.220450 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.220427    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/783fffe8-eda8-419f-ac06-320afd1ab891-metrics-cert\") pod \"opendatahub-operator-controller-manager-899c649cb-gs59t\" (UID: \"783fffe8-eda8-419f-ac06-320afd1ab891\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-899c649cb-gs59t"
Oct 31 21:21:12.245204 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.245179    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-znj4h\" (UniqueName: \"kubernetes.io/projected/783fffe8-eda8-419f-ac06-320afd1ab891-kube-api-access-znj4h\") pod \"opendatahub-operator-controller-manager-899c649cb-gs59t\" (UID: \"783fffe8-eda8-419f-ac06-320afd1ab891\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-899c649cb-gs59t"
Oct 31 21:21:12.331742 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.331706    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:12.393740 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.393658    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-899c649cb-gs59t"
Oct 31 21:21:12.473262 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.473229    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:12.588359 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.588320    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:12.752091 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.752015    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-899c649cb-gs59t"]
Oct 31 21:21:12.755462 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:21:12.755435    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod783fffe8_eda8_419f_ac06_320afd1ab891.slice/crio-71779b410652af6e52edeb91030189920e7cd5074a3fc9ec38e8f6754a010ba9 WatchSource:0}: Error finding container 71779b410652af6e52edeb91030189920e7cd5074a3fc9ec38e8f6754a010ba9: Status 404 returned error can't find the container with id 71779b410652af6e52edeb91030189920e7cd5074a3fc9ec38e8f6754a010ba9
Oct 31 21:21:12.757030 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:12.757009    4460 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 21:21:13.385192 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:13.385156    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:13.482689 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:13.482594    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-899c649cb-gs59t" event={"ID":"783fffe8-eda8-419f-ac06-320afd1ab891","Type":"ContainerStarted","Data":"71779b410652af6e52edeb91030189920e7cd5074a3fc9ec38e8f6754a010ba9"}
Oct 31 21:21:14.525203 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:14.525167    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:16.366789 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:16.366753    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:16.498659 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:16.498607    4460 generic.go:356] "Generic (PLEG): container finished" podID="783fffe8-eda8-419f-ac06-320afd1ab891" containerID="c4dbcf13f70609721013b5f28c5d6adcb878c00b58ebdbbe7fb6ea5034d7c650" exitCode=0
Oct 31 21:21:16.498824 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:16.498695    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-899c649cb-gs59t" event={"ID":"783fffe8-eda8-419f-ac06-320afd1ab891","Type":"ContainerDied","Data":"c4dbcf13f70609721013b5f28c5d6adcb878c00b58ebdbbe7fb6ea5034d7c650"}
Oct 31 21:21:17.504653 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:17.504617    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-899c649cb-gs59t" event={"ID":"783fffe8-eda8-419f-ac06-320afd1ab891","Type":"ContainerStarted","Data":"bf1735fe672283b92b8f103d7ed7cbc309a3d07ca72638c81df52231b4c47558"}
Oct 31 21:21:17.505067 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:17.504755    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-899c649cb-gs59t"
Oct 31 21:21:17.598734 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:17.598677    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-899c649cb-gs59t" podStartSLOduration=2.146223598 podStartE2EDuration="5.598662803s" totalImagesPullingTime="3.021921335s" totalInitContainerRuntime="430.51787ms" isStatefulPod=false podCreationTimestamp="2026-10-31 21:21:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:21:17.584136948 +0000 UTC m=+317.500446373" watchObservedRunningTime="2026-10-31 21:21:17.598662803 +0000 UTC m=+317.514972233"
Oct 31 21:21:18.107025 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:18.106986    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:18.266221 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:18.266188    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:19.747661 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:19.747623    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:20.338151 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:20.338112    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:20.793612 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:20.793565    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:20.955609 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:20.955574    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:21.204548 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:21.204451    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:21.451846 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:21:21.451800    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:21:21.967260 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:21.967210    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:22.114231 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:22.114190    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:25.532268 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:25.532230    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:25.768832 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:25.768802    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:25.909261 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:25.909184    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:27.528648 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:27.528615    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:27.986316 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:27.986223    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:28.512187 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:28.512161    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-899c649cb-gs59t"
Oct 31 21:21:31.314240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:31.314207    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:31.970335 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:31.970299    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:34.247109 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:34.247078    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:34.464923 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:34.464889    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:34.595301 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:34.595269    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:34.783063 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:34.783028    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:36.398456 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:36.398414    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:36.530049 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:36.530012    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:36.954315 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:36.954282    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:37.073995 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:37.073958    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:37.192442 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:37.192414    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:37.381845 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:37.381809    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-5w9v5"]
Oct 31 21:21:37.385574 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:37.385557    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-5w9v5"
Oct 31 21:21:37.404805 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:37.404784    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-5w9v5"]
Oct 31 21:21:37.417540 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:37.417522    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-ppbt2\""
Oct 31 21:21:37.489534 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:37.489507    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Oct 31 21:21:37.489767 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:37.489751    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Oct 31 21:21:37.535174 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:37.535146    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/47ac2aae-5ff3-4c79-9f52-967aa39719ed-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-5w9v5\" (UID: \"47ac2aae-5ff3-4c79-9f52-967aa39719ed\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-5w9v5"
Oct 31 21:21:37.535295 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:37.535190    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ljgj\" (UniqueName: \"kubernetes.io/projected/47ac2aae-5ff3-4c79-9f52-967aa39719ed-kube-api-access-4ljgj\") pod \"cert-manager-cainjector-5c57bbd4fc-5w9v5\" (UID: \"47ac2aae-5ff3-4c79-9f52-967aa39719ed\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-5w9v5"
Oct 31 21:21:37.636425 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:37.636352    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/47ac2aae-5ff3-4c79-9f52-967aa39719ed-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-5w9v5\" (UID: \"47ac2aae-5ff3-4c79-9f52-967aa39719ed\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-5w9v5"
Oct 31 21:21:37.636425 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:37.636397    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ljgj\" (UniqueName: \"kubernetes.io/projected/47ac2aae-5ff3-4c79-9f52-967aa39719ed-kube-api-access-4ljgj\") pod \"cert-manager-cainjector-5c57bbd4fc-5w9v5\" (UID: \"47ac2aae-5ff3-4c79-9f52-967aa39719ed\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-5w9v5"
Oct 31 21:21:37.661504 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:37.661468    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ljgj\" (UniqueName: \"kubernetes.io/projected/47ac2aae-5ff3-4c79-9f52-967aa39719ed-kube-api-access-4ljgj\") pod \"cert-manager-cainjector-5c57bbd4fc-5w9v5\" (UID: \"47ac2aae-5ff3-4c79-9f52-967aa39719ed\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-5w9v5"
Oct 31 21:21:37.662725 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:37.662705    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/47ac2aae-5ff3-4c79-9f52-967aa39719ed-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-5w9v5\" (UID: \"47ac2aae-5ff3-4c79-9f52-967aa39719ed\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-5w9v5"
Oct 31 21:21:37.707319 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:37.707296    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-5w9v5"
Oct 31 21:21:38.003604 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:38.003578    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-5w9v5"]
Oct 31 21:21:38.005477 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:21:38.005438    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod47ac2aae_5ff3_4c79_9f52_967aa39719ed.slice/crio-b68c1ff64322943a8c036d712a0d2e70fa78a22f054b2cdbb77f6d3cace984eb WatchSource:0}: Error finding container b68c1ff64322943a8c036d712a0d2e70fa78a22f054b2cdbb77f6d3cace984eb: Status 404 returned error can't find the container with id b68c1ff64322943a8c036d712a0d2e70fa78a22f054b2cdbb77f6d3cace984eb
Oct 31 21:21:38.483396 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:38.483363    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:38.588019 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:38.587979    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-5w9v5" event={"ID":"47ac2aae-5ff3-4c79-9f52-967aa39719ed","Type":"ContainerStarted","Data":"b68c1ff64322943a8c036d712a0d2e70fa78a22f054b2cdbb77f6d3cace984eb"}
Oct 31 21:21:38.593057 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:38.593020    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:38.901638 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:38.901601    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:39.319558 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:39.319465    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:39.448084 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:39.447986    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:39.647471 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:39.647378    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:39.782257 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:39.782225    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:40.894100 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:40.894016    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:41.603991 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:41.603955    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-5w9v5" event={"ID":"47ac2aae-5ff3-4c79-9f52-967aa39719ed","Type":"ContainerStarted","Data":"5532c8a4728632193cfa63caa221bc96ab4c8c55bbb6ad6f837513e59dd1b742"}
Oct 31 21:21:41.649962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:41.649895    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-5w9v5" podStartSLOduration=1.6632754699999999 podStartE2EDuration="4.649881077s" totalImagesPullingTime="2.986605607s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:21:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:21:41.648649793 +0000 UTC m=+341.564959218" watchObservedRunningTime="2026-10-31 21:21:41.649881077 +0000 UTC m=+341.566190505"
Oct 31 21:21:42.064797 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:42.064760    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:42.185119 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:42.185082    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:42.246204 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:42.246173    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-sh9g6"]
Oct 31 21:21:42.248685 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:42.248661    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-sh9g6"
Oct 31 21:21:42.280071 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:42.280041    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Oct 31 21:21:42.280217 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:42.280191    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 31 21:21:42.280285 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:42.280213    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kdzsw\""
Oct 31 21:21:42.280285 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:42.280273    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 31 21:21:42.281621 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:42.281602    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-sh9g6"]
Oct 31 21:21:42.305701 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:42.305673    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:42.379984 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:42.379901    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/826d904a-89e5-4927-a7cf-cc8930d95288-config-file\") pod \"limitador-limitador-74dfcfb894-sh9g6\" (UID: \"826d904a-89e5-4927-a7cf-cc8930d95288\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-sh9g6"
Oct 31 21:21:42.380142 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:42.380049    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m97hw\" (UniqueName: \"kubernetes.io/projected/826d904a-89e5-4927-a7cf-cc8930d95288-kube-api-access-m97hw\") pod \"limitador-limitador-74dfcfb894-sh9g6\" (UID: \"826d904a-89e5-4927-a7cf-cc8930d95288\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-sh9g6"
Oct 31 21:21:42.389278 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:42.389253    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-sh9g6"]
Oct 31 21:21:42.434477 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:42.434442    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:42.480881 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:42.480851    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m97hw\" (UniqueName: \"kubernetes.io/projected/826d904a-89e5-4927-a7cf-cc8930d95288-kube-api-access-m97hw\") pod \"limitador-limitador-74dfcfb894-sh9g6\" (UID: \"826d904a-89e5-4927-a7cf-cc8930d95288\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-sh9g6"
Oct 31 21:21:42.481090 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:42.480903    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/826d904a-89e5-4927-a7cf-cc8930d95288-config-file\") pod \"limitador-limitador-74dfcfb894-sh9g6\" (UID: \"826d904a-89e5-4927-a7cf-cc8930d95288\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-sh9g6"
Oct 31 21:21:42.481579 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:42.481560    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/826d904a-89e5-4927-a7cf-cc8930d95288-config-file\") pod \"limitador-limitador-74dfcfb894-sh9g6\" (UID: \"826d904a-89e5-4927-a7cf-cc8930d95288\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-sh9g6"
Oct 31 21:21:42.513080 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:42.513050    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m97hw\" (UniqueName: \"kubernetes.io/projected/826d904a-89e5-4927-a7cf-cc8930d95288-kube-api-access-m97hw\") pod \"limitador-limitador-74dfcfb894-sh9g6\" (UID: \"826d904a-89e5-4927-a7cf-cc8930d95288\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-sh9g6"
Oct 31 21:21:42.549231 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:42.549200    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:42.560912 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:42.560888    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-sh9g6"
Oct 31 21:21:42.658142 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:42.658058    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:42.787685 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:42.787654    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:21:42.790271 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:42.790255    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"
Oct 31 21:21:42.808092 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:42.808065    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:21:42.874681 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:42.874659    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-sh9g6"]
Oct 31 21:21:42.877213 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:21:42.877186    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod826d904a_89e5_4927_a7cf_cc8930d95288.slice/crio-b6d92fe06bf2461a2d9a1028d7204438a844f285b80495983df2c6574bf8cba8 WatchSource:0}: Error finding container b6d92fe06bf2461a2d9a1028d7204438a844f285b80495983df2c6574bf8cba8: Status 404 returned error can't find the container with id b6d92fe06bf2461a2d9a1028d7204438a844f285b80495983df2c6574bf8cba8
Oct 31 21:21:42.883298 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:42.883276    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:21:42.886021 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:42.885999    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srfxc\" (UniqueName: \"kubernetes.io/projected/7fd24cb1-b574-4cfb-8d3c-ee13bae2e73e-kube-api-access-srfxc\") pod \"limitador-limitador-7c9b87d574-8p5kz\" (UID: \"7fd24cb1-b574-4cfb-8d3c-ee13bae2e73e\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"
Oct 31 21:21:42.886129 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:42.886042    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7fd24cb1-b574-4cfb-8d3c-ee13bae2e73e-config-file\") pod \"limitador-limitador-7c9b87d574-8p5kz\" (UID: \"7fd24cb1-b574-4cfb-8d3c-ee13bae2e73e\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"
Oct 31 21:21:42.932061 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:42.931998    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:42.986555 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:42.986531    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7fd24cb1-b574-4cfb-8d3c-ee13bae2e73e-config-file\") pod \"limitador-limitador-7c9b87d574-8p5kz\" (UID: \"7fd24cb1-b574-4cfb-8d3c-ee13bae2e73e\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"
Oct 31 21:21:42.986651 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:42.986636    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srfxc\" (UniqueName: \"kubernetes.io/projected/7fd24cb1-b574-4cfb-8d3c-ee13bae2e73e-kube-api-access-srfxc\") pod \"limitador-limitador-7c9b87d574-8p5kz\" (UID: \"7fd24cb1-b574-4cfb-8d3c-ee13bae2e73e\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"
Oct 31 21:21:42.987261 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:42.987244    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7fd24cb1-b574-4cfb-8d3c-ee13bae2e73e-config-file\") pod \"limitador-limitador-7c9b87d574-8p5kz\" (UID: \"7fd24cb1-b574-4cfb-8d3c-ee13bae2e73e\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"
Oct 31 21:21:43.012173 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:43.012152    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-srfxc\" (UniqueName: \"kubernetes.io/projected/7fd24cb1-b574-4cfb-8d3c-ee13bae2e73e-kube-api-access-srfxc\") pod \"limitador-limitador-7c9b87d574-8p5kz\" (UID: \"7fd24cb1-b574-4cfb-8d3c-ee13bae2e73e\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"
Oct 31 21:21:43.048419 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:43.048382    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:43.103028 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:43.103002    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"
Oct 31 21:21:43.240788 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:43.240716    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:43.357157 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:43.357128    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:43.398438 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:43.398413    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:21:43.400798 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:21:43.400773    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fd24cb1_b574_4cfb_8d3c_ee13bae2e73e.slice/crio-91930bba9435774e0c143e34a29b224572c5906a0271175b013d46353e315989 WatchSource:0}: Error finding container 91930bba9435774e0c143e34a29b224572c5906a0271175b013d46353e315989: Status 404 returned error can't find the container with id 91930bba9435774e0c143e34a29b224572c5906a0271175b013d46353e315989
Oct 31 21:21:43.488987 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:43.488955    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:43.617480 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:43.617444    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-8p5kz" event={"ID":"7fd24cb1-b574-4cfb-8d3c-ee13bae2e73e","Type":"ContainerStarted","Data":"91930bba9435774e0c143e34a29b224572c5906a0271175b013d46353e315989"}
Oct 31 21:21:43.618874 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:43.618845    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-sh9g6" event={"ID":"826d904a-89e5-4927-a7cf-cc8930d95288","Type":"ContainerStarted","Data":"b6d92fe06bf2461a2d9a1028d7204438a844f285b80495983df2c6574bf8cba8"}
Oct 31 21:21:44.504442 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:44.504409    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:45.629278 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:45.629189    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-sh9g6" event={"ID":"826d904a-89e5-4927-a7cf-cc8930d95288","Type":"ContainerStarted","Data":"63db7e700ccc18b4c7833ff110ff28c72d6f1f6112ec4772da02bea8a6dac967"}
Oct 31 21:21:45.629700 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:45.629279    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-sh9g6"
Oct 31 21:21:45.630809 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:45.630785    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-8p5kz" event={"ID":"7fd24cb1-b574-4cfb-8d3c-ee13bae2e73e","Type":"ContainerStarted","Data":"0bd1391ad8906ccb6f6076dbfc277779e01dcf66fd624258982321683c3dfee7"}
Oct 31 21:21:45.630920 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:45.630848    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"
Oct 31 21:21:45.675051 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:45.674987    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-sh9g6" podStartSLOduration=1.247030297 podStartE2EDuration="3.674974659s" totalImagesPullingTime="2.427944362s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:21:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:21:45.673581324 +0000 UTC m=+345.589890753" watchObservedRunningTime="2026-10-31 21:21:45.674974659 +0000 UTC m=+345.591284100"
Oct 31 21:21:45.718706 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:45.718639    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7c9b87d574-8p5kz" podStartSLOduration=1.724288535 podStartE2EDuration="3.718626973s" totalImagesPullingTime="1.994338438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:21:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:21:45.718332725 +0000 UTC m=+345.634642153" watchObservedRunningTime="2026-10-31 21:21:45.718626973 +0000 UTC m=+345.634936401"
Oct 31 21:21:47.475482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:47.475449    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-q7c24"]
Oct 31 21:21:47.478659 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:47.478637    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-q7c24"
Oct 31 21:21:47.497057 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:47.497021    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-q7c24"]
Oct 31 21:21:47.509323 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:47.509302    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-mdhqv\""
Oct 31 21:21:47.631358 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:47.631319    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kv2lr\" (UniqueName: \"kubernetes.io/projected/851a9930-c941-4210-81a4-420e371b29fa-kube-api-access-kv2lr\") pod \"cert-manager-597b95cb84-q7c24\" (UID: \"851a9930-c941-4210-81a4-420e371b29fa\") " pod="cert-manager/cert-manager-597b95cb84-q7c24"
Oct 31 21:21:47.631528 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:47.631376    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/851a9930-c941-4210-81a4-420e371b29fa-bound-sa-token\") pod \"cert-manager-597b95cb84-q7c24\" (UID: \"851a9930-c941-4210-81a4-420e371b29fa\") " pod="cert-manager/cert-manager-597b95cb84-q7c24"
Oct 31 21:21:47.732587 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:47.732516    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kv2lr\" (UniqueName: \"kubernetes.io/projected/851a9930-c941-4210-81a4-420e371b29fa-kube-api-access-kv2lr\") pod \"cert-manager-597b95cb84-q7c24\" (UID: \"851a9930-c941-4210-81a4-420e371b29fa\") " pod="cert-manager/cert-manager-597b95cb84-q7c24"
Oct 31 21:21:47.732587 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:47.732572    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/851a9930-c941-4210-81a4-420e371b29fa-bound-sa-token\") pod \"cert-manager-597b95cb84-q7c24\" (UID: \"851a9930-c941-4210-81a4-420e371b29fa\") " pod="cert-manager/cert-manager-597b95cb84-q7c24"
Oct 31 21:21:47.757930 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:47.757888    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/851a9930-c941-4210-81a4-420e371b29fa-bound-sa-token\") pod \"cert-manager-597b95cb84-q7c24\" (UID: \"851a9930-c941-4210-81a4-420e371b29fa\") " pod="cert-manager/cert-manager-597b95cb84-q7c24"
Oct 31 21:21:47.758114 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:47.758020    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kv2lr\" (UniqueName: \"kubernetes.io/projected/851a9930-c941-4210-81a4-420e371b29fa-kube-api-access-kv2lr\") pod \"cert-manager-597b95cb84-q7c24\" (UID: \"851a9930-c941-4210-81a4-420e371b29fa\") " pod="cert-manager/cert-manager-597b95cb84-q7c24"
Oct 31 21:21:47.789752 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:47.789717    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-q7c24"
Oct 31 21:21:48.093583 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:48.093557    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-q7c24"]
Oct 31 21:21:48.096233 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:21:48.096208    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod851a9930_c941_4210_81a4_420e371b29fa.slice/crio-b63006ec3ff9c018ec206132432afab5e61e1d6cbedfdf8266f6f2fc0889bbeb WatchSource:0}: Error finding container b63006ec3ff9c018ec206132432afab5e61e1d6cbedfdf8266f6f2fc0889bbeb: Status 404 returned error can't find the container with id b63006ec3ff9c018ec206132432afab5e61e1d6cbedfdf8266f6f2fc0889bbeb
Oct 31 21:21:48.394586 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:48.394488    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:48.503811 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:48.503780    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:48.644006 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:48.643970    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-q7c24" event={"ID":"851a9930-c941-4210-81a4-420e371b29fa","Type":"ContainerStarted","Data":"d2b554470ea636ac6f2a050c687bf3ada6b564a6adff34899a8dbb71084a4a3e"}
Oct 31 21:21:48.644169 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:48.644014    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-q7c24" event={"ID":"851a9930-c941-4210-81a4-420e371b29fa","Type":"ContainerStarted","Data":"b63006ec3ff9c018ec206132432afab5e61e1d6cbedfdf8266f6f2fc0889bbeb"}
Oct 31 21:21:48.687720 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:48.687632    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-q7c24" podStartSLOduration=1.687619076 podStartE2EDuration="1.687619076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:21:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:21:48.687226526 +0000 UTC m=+348.603535953" watchObservedRunningTime="2026-10-31 21:21:48.687619076 +0000 UTC m=+348.603928501"
Oct 31 21:21:51.449746 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:21:51.449711    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:21:53.149065 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:53.149028    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:53.265159 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:53.265116    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:56.124614 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:56.124577    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:56.235989 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:56.235932    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:21:56.635718 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:56.635687    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"
Oct 31 21:21:56.635885 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:56.635758    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-sh9g6"
Oct 31 21:21:56.708651 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:56.708619    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-sh9g6"]
Oct 31 21:21:56.708821 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:56.708802    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-sh9g6" podUID="826d904a-89e5-4927-a7cf-cc8930d95288" containerName="limitador" containerID="cri-o://63db7e700ccc18b4c7833ff110ff28c72d6f1f6112ec4772da02bea8a6dac967" gracePeriod=30
Oct 31 21:21:57.247210 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:57.247185    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-sh9g6"
Oct 31 21:21:57.307141 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:57.307077    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/826d904a-89e5-4927-a7cf-cc8930d95288-config-file\" (UniqueName: \"kubernetes.io/configmap/826d904a-89e5-4927-a7cf-cc8930d95288-config-file\") pod \"826d904a-89e5-4927-a7cf-cc8930d95288\" (UID: \"826d904a-89e5-4927-a7cf-cc8930d95288\") "
Oct 31 21:21:57.307141 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:57.307110    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/826d904a-89e5-4927-a7cf-cc8930d95288-kube-api-access-m97hw\" (UniqueName: \"kubernetes.io/projected/826d904a-89e5-4927-a7cf-cc8930d95288-kube-api-access-m97hw\") pod \"826d904a-89e5-4927-a7cf-cc8930d95288\" (UID: \"826d904a-89e5-4927-a7cf-cc8930d95288\") "
Oct 31 21:21:57.312108 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:57.307820    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/826d904a-89e5-4927-a7cf-cc8930d95288-config-file" pod "826d904a-89e5-4927-a7cf-cc8930d95288" (UID: "826d904a-89e5-4927-a7cf-cc8930d95288"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:21:57.312108 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:57.308076    4460 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/826d904a-89e5-4927-a7cf-cc8930d95288-config-file\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:21:57.314648 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:57.314620    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/826d904a-89e5-4927-a7cf-cc8930d95288-kube-api-access-m97hw" pod "826d904a-89e5-4927-a7cf-cc8930d95288" (UID: "826d904a-89e5-4927-a7cf-cc8930d95288"). InnerVolumeSpecName "kube-api-access-m97hw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:21:57.408652 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:57.408624    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m97hw\" (UniqueName: \"kubernetes.io/projected/826d904a-89e5-4927-a7cf-cc8930d95288-kube-api-access-m97hw\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:21:57.681810 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:57.681732    4460 generic.go:356] "Generic (PLEG): container finished" podID="826d904a-89e5-4927-a7cf-cc8930d95288" containerID="63db7e700ccc18b4c7833ff110ff28c72d6f1f6112ec4772da02bea8a6dac967" exitCode=0
Oct 31 21:21:57.681967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:57.681808    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-sh9g6"
Oct 31 21:21:57.681967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:57.681820    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-sh9g6" event={"ID":"826d904a-89e5-4927-a7cf-cc8930d95288","Type":"ContainerDied","Data":"63db7e700ccc18b4c7833ff110ff28c72d6f1f6112ec4772da02bea8a6dac967"}
Oct 31 21:21:57.681967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:57.681922    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-sh9g6" event={"ID":"826d904a-89e5-4927-a7cf-cc8930d95288","Type":"ContainerDied","Data":"b6d92fe06bf2461a2d9a1028d7204438a844f285b80495983df2c6574bf8cba8"}
Oct 31 21:21:57.681967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:57.681965    4460 scope.go:122] "RemoveContainer" containerID="63db7e700ccc18b4c7833ff110ff28c72d6f1f6112ec4772da02bea8a6dac967"
Oct 31 21:21:57.690074 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:57.690058    4460 scope.go:122] "RemoveContainer" containerID="63db7e700ccc18b4c7833ff110ff28c72d6f1f6112ec4772da02bea8a6dac967"
Oct 31 21:21:57.690294 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:21:57.690276    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63db7e700ccc18b4c7833ff110ff28c72d6f1f6112ec4772da02bea8a6dac967\": container with ID starting with 63db7e700ccc18b4c7833ff110ff28c72d6f1f6112ec4772da02bea8a6dac967 not found: ID does not exist" containerID="63db7e700ccc18b4c7833ff110ff28c72d6f1f6112ec4772da02bea8a6dac967"
Oct 31 21:21:57.690351 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:57.690303    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63db7e700ccc18b4c7833ff110ff28c72d6f1f6112ec4772da02bea8a6dac967"} err="failed to get container status \"63db7e700ccc18b4c7833ff110ff28c72d6f1f6112ec4772da02bea8a6dac967\": rpc error: code = NotFound desc = could not find container \"63db7e700ccc18b4c7833ff110ff28c72d6f1f6112ec4772da02bea8a6dac967\": container with ID starting with 63db7e700ccc18b4c7833ff110ff28c72d6f1f6112ec4772da02bea8a6dac967 not found: ID does not exist"
Oct 31 21:21:57.736869 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:57.736845    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-sh9g6"]
Oct 31 21:21:57.750598 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:57.750567    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-sh9g6"]
Oct 31 21:21:58.222272 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:21:58.222240    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="826d904a-89e5-4927-a7cf-cc8930d95288" path="/var/lib/kubelet/pods/826d904a-89e5-4927-a7cf-cc8930d95288/volumes"
Oct 31 21:22:00.845154 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:00.845128    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:00.953359 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:00.953323    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:01.357255 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:01.357219    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:01.466652 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:01.466605    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:01.608773 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:01.608700    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:01.743709 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:01.743671    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:01.855323 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:01.855281    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:01.967677 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:01.967596    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:07.521210 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:07.521163    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:07.631998 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:07.631958    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:12.271993 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:12.271929    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:12.394317 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:12.394277    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:17.967451 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:17.967418    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:18.081795 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:18.081756    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:18.702421 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:18.702388    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-667b7b4d9b-znklp"]
Oct 31 21:22:18.702792 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:18.702775    4460 cpu_manager.go:414] "containerMap: removing container" podUID="826d904a-89e5-4927-a7cf-cc8930d95288" containerName="limitador"
Oct 31 21:22:18.702837 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:18.702793    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="826d904a-89e5-4927-a7cf-cc8930d95288" containerName="limitador"
Oct 31 21:22:18.702871 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:18.702862    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="826d904a-89e5-4927-a7cf-cc8930d95288" containerName="limitador"
Oct 31 21:22:18.709395 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:18.709381    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-667b7b4d9b-znklp"
Oct 31 21:22:18.726570 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:18.726541    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-667b7b4d9b-znklp"]
Oct 31 21:22:18.738249 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:18.738228    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"webhook-server-cert\""
Oct 31 21:22:18.738379 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:18.738358    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"metrics-server-cert\""
Oct 31 21:22:18.739300 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:18.739285    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Oct 31 21:22:18.739374 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:18.739324    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Oct 31 21:22:18.739426 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:18.739383    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"jobset-manager-config\""
Oct 31 21:22:18.739788 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:18.739776    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-controller-manager-dockercfg-rlkpj\""
Oct 31 21:22:18.794901 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:18.794872    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54x7g\" (UniqueName: \"kubernetes.io/projected/b545f7a3-980a-4ba3-a385-2a9874cbef71-kube-api-access-54x7g\") pod \"jobset-controller-manager-667b7b4d9b-znklp\" (UID: \"b545f7a3-980a-4ba3-a385-2a9874cbef71\") " pod="openshift-jobset-operator/jobset-controller-manager-667b7b4d9b-znklp"
Oct 31 21:22:18.794901 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:18.794910    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b545f7a3-980a-4ba3-a385-2a9874cbef71-cert\") pod \"jobset-controller-manager-667b7b4d9b-znklp\" (UID: \"b545f7a3-980a-4ba3-a385-2a9874cbef71\") " pod="openshift-jobset-operator/jobset-controller-manager-667b7b4d9b-znklp"
Oct 31 21:22:18.795102 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:18.795008    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b545f7a3-980a-4ba3-a385-2a9874cbef71-metrics-certs\") pod \"jobset-controller-manager-667b7b4d9b-znklp\" (UID: \"b545f7a3-980a-4ba3-a385-2a9874cbef71\") " pod="openshift-jobset-operator/jobset-controller-manager-667b7b4d9b-znklp"
Oct 31 21:22:18.795102 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:18.795060    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b545f7a3-980a-4ba3-a385-2a9874cbef71-manager-config\") pod \"jobset-controller-manager-667b7b4d9b-znklp\" (UID: \"b545f7a3-980a-4ba3-a385-2a9874cbef71\") " pod="openshift-jobset-operator/jobset-controller-manager-667b7b4d9b-znklp"
Oct 31 21:22:18.895746 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:18.895713    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54x7g\" (UniqueName: \"kubernetes.io/projected/b545f7a3-980a-4ba3-a385-2a9874cbef71-kube-api-access-54x7g\") pod \"jobset-controller-manager-667b7b4d9b-znklp\" (UID: \"b545f7a3-980a-4ba3-a385-2a9874cbef71\") " pod="openshift-jobset-operator/jobset-controller-manager-667b7b4d9b-znklp"
Oct 31 21:22:18.895746 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:18.895751    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b545f7a3-980a-4ba3-a385-2a9874cbef71-cert\") pod \"jobset-controller-manager-667b7b4d9b-znklp\" (UID: \"b545f7a3-980a-4ba3-a385-2a9874cbef71\") " pod="openshift-jobset-operator/jobset-controller-manager-667b7b4d9b-znklp"
Oct 31 21:22:18.896007 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:18.895775    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b545f7a3-980a-4ba3-a385-2a9874cbef71-metrics-certs\") pod \"jobset-controller-manager-667b7b4d9b-znklp\" (UID: \"b545f7a3-980a-4ba3-a385-2a9874cbef71\") " pod="openshift-jobset-operator/jobset-controller-manager-667b7b4d9b-znklp"
Oct 31 21:22:18.896007 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:18.895809    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b545f7a3-980a-4ba3-a385-2a9874cbef71-manager-config\") pod \"jobset-controller-manager-667b7b4d9b-znklp\" (UID: \"b545f7a3-980a-4ba3-a385-2a9874cbef71\") " pod="openshift-jobset-operator/jobset-controller-manager-667b7b4d9b-znklp"
Oct 31 21:22:18.896518 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:18.896491    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b545f7a3-980a-4ba3-a385-2a9874cbef71-manager-config\") pod \"jobset-controller-manager-667b7b4d9b-znklp\" (UID: \"b545f7a3-980a-4ba3-a385-2a9874cbef71\") " pod="openshift-jobset-operator/jobset-controller-manager-667b7b4d9b-znklp"
Oct 31 21:22:18.898369 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:18.898342    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b545f7a3-980a-4ba3-a385-2a9874cbef71-cert\") pod \"jobset-controller-manager-667b7b4d9b-znklp\" (UID: \"b545f7a3-980a-4ba3-a385-2a9874cbef71\") " pod="openshift-jobset-operator/jobset-controller-manager-667b7b4d9b-znklp"
Oct 31 21:22:18.898454 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:18.898342    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b545f7a3-980a-4ba3-a385-2a9874cbef71-metrics-certs\") pod \"jobset-controller-manager-667b7b4d9b-znklp\" (UID: \"b545f7a3-980a-4ba3-a385-2a9874cbef71\") " pod="openshift-jobset-operator/jobset-controller-manager-667b7b4d9b-znklp"
Oct 31 21:22:18.920614 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:18.920590    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-54x7g\" (UniqueName: \"kubernetes.io/projected/b545f7a3-980a-4ba3-a385-2a9874cbef71-kube-api-access-54x7g\") pod \"jobset-controller-manager-667b7b4d9b-znklp\" (UID: \"b545f7a3-980a-4ba3-a385-2a9874cbef71\") " pod="openshift-jobset-operator/jobset-controller-manager-667b7b4d9b-znklp"
Oct 31 21:22:19.021521 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:19.021491    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-667b7b4d9b-znklp"
Oct 31 21:22:19.334717 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:19.334692    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-667b7b4d9b-znklp"]
Oct 31 21:22:19.337257 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:22:19.337229    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb545f7a3_980a_4ba3_a385_2a9874cbef71.slice/crio-730ada5726db31aed2d5f2f9126467489a44b82761f7539a7ff4d760382fbd2e WatchSource:0}: Error finding container 730ada5726db31aed2d5f2f9126467489a44b82761f7539a7ff4d760382fbd2e: Status 404 returned error can't find the container with id 730ada5726db31aed2d5f2f9126467489a44b82761f7539a7ff4d760382fbd2e
Oct 31 21:22:19.780259 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:19.780224    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-667b7b4d9b-znklp" event={"ID":"b545f7a3-980a-4ba3-a385-2a9874cbef71","Type":"ContainerStarted","Data":"730ada5726db31aed2d5f2f9126467489a44b82761f7539a7ff4d760382fbd2e"}
Oct 31 21:22:20.429697 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:20.429657    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:20.547719 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:20.547685    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:20.659274 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:20.659235    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:20.767749 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:20.767714    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:21.452160 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:22:21.452120    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:22:22.794551 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:22.794516    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-667b7b4d9b-znklp" event={"ID":"b545f7a3-980a-4ba3-a385-2a9874cbef71","Type":"ContainerStarted","Data":"e75e3cef7347f09e6b916096198a9d540550189e55a508f4cc0dd876502c75a6"}
Oct 31 21:22:22.794992 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:22.794606    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-jobset-operator/jobset-controller-manager-667b7b4d9b-znklp"
Oct 31 21:22:22.841496 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:22.841439    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-controller-manager-667b7b4d9b-znklp" podStartSLOduration=2.353329449 podStartE2EDuration="4.841422366s" totalImagesPullingTime="2.488092917s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:22:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:22:22.840740632 +0000 UTC m=+382.757050063" watchObservedRunningTime="2026-10-31 21:22:22.841422366 +0000 UTC m=+382.757731795"
Oct 31 21:22:24.233898 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:24.233863    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:24.353130 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:24.353097    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:25.861081 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:25.861043    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:26.083170 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:26.083125    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:28.278556 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:28.278522    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:28.388697 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:28.388652    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:33.804831 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:33.804798    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-jobset-operator/jobset-controller-manager-667b7b4d9b-znklp"
Oct 31 21:22:43.631284 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:43.631202    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:46.304499 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:46.304465    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dd27"]
Oct 31 21:22:46.307687 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:46.307664    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dd27"
Oct 31 21:22:46.338832 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:46.338802    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1c463ce3-50ac-473d-bb96-ebc5c3c90e6d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dd27\" (UID: \"1c463ce3-50ac-473d-bb96-ebc5c3c90e6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dd27"
Oct 31 21:22:46.338992 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:46.338838    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c463ce3-50ac-473d-bb96-ebc5c3c90e6d-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dd27\" (UID: \"1c463ce3-50ac-473d-bb96-ebc5c3c90e6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dd27"
Oct 31 21:22:46.338992 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:46.338862    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c463ce3-50ac-473d-bb96-ebc5c3c90e6d-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dd27\" (UID: \"1c463ce3-50ac-473d-bb96-ebc5c3c90e6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dd27"
Oct 31 21:22:46.339121 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:46.339067    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1c463ce3-50ac-473d-bb96-ebc5c3c90e6d-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dd27\" (UID: \"1c463ce3-50ac-473d-bb96-ebc5c3c90e6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dd27"
Oct 31 21:22:46.339121 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:46.339107    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1c463ce3-50ac-473d-bb96-ebc5c3c90e6d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dd27\" (UID: \"1c463ce3-50ac-473d-bb96-ebc5c3c90e6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dd27"
Oct 31 21:22:46.339236 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:46.339195    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1c463ce3-50ac-473d-bb96-ebc5c3c90e6d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dd27\" (UID: \"1c463ce3-50ac-473d-bb96-ebc5c3c90e6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dd27"
Oct 31 21:22:46.339236 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:46.339225    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7pqj\" (UniqueName: \"kubernetes.io/projected/1c463ce3-50ac-473d-bb96-ebc5c3c90e6d-kube-api-access-k7pqj\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dd27\" (UID: \"1c463ce3-50ac-473d-bb96-ebc5c3c90e6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dd27"
Oct 31 21:22:46.341318 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:46.341293    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dd27"]
Oct 31 21:22:46.440439 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:46.440406    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1c463ce3-50ac-473d-bb96-ebc5c3c90e6d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dd27\" (UID: \"1c463ce3-50ac-473d-bb96-ebc5c3c90e6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dd27"
Oct 31 21:22:46.440616 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:46.440448    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7pqj\" (UniqueName: \"kubernetes.io/projected/1c463ce3-50ac-473d-bb96-ebc5c3c90e6d-kube-api-access-k7pqj\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dd27\" (UID: \"1c463ce3-50ac-473d-bb96-ebc5c3c90e6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dd27"
Oct 31 21:22:46.440616 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:46.440490    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1c463ce3-50ac-473d-bb96-ebc5c3c90e6d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dd27\" (UID: \"1c463ce3-50ac-473d-bb96-ebc5c3c90e6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dd27"
Oct 31 21:22:46.440616 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:46.440521    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c463ce3-50ac-473d-bb96-ebc5c3c90e6d-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dd27\" (UID: \"1c463ce3-50ac-473d-bb96-ebc5c3c90e6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dd27"
Oct 31 21:22:46.440616 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:46.440553    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c463ce3-50ac-473d-bb96-ebc5c3c90e6d-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dd27\" (UID: \"1c463ce3-50ac-473d-bb96-ebc5c3c90e6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dd27"
Oct 31 21:22:46.440806 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:46.440693    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1c463ce3-50ac-473d-bb96-ebc5c3c90e6d-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dd27\" (UID: \"1c463ce3-50ac-473d-bb96-ebc5c3c90e6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dd27"
Oct 31 21:22:46.440806 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:46.440737    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1c463ce3-50ac-473d-bb96-ebc5c3c90e6d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dd27\" (UID: \"1c463ce3-50ac-473d-bb96-ebc5c3c90e6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dd27"
Oct 31 21:22:46.442425 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:46.441957    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1c463ce3-50ac-473d-bb96-ebc5c3c90e6d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dd27\" (UID: \"1c463ce3-50ac-473d-bb96-ebc5c3c90e6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dd27"
Oct 31 21:22:46.446760 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:46.444107    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1c463ce3-50ac-473d-bb96-ebc5c3c90e6d-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dd27\" (UID: \"1c463ce3-50ac-473d-bb96-ebc5c3c90e6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dd27"
Oct 31 21:22:46.449824 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:46.449780    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1c463ce3-50ac-473d-bb96-ebc5c3c90e6d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dd27\" (UID: \"1c463ce3-50ac-473d-bb96-ebc5c3c90e6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dd27"
Oct 31 21:22:46.450062 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:46.450039    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c463ce3-50ac-473d-bb96-ebc5c3c90e6d-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dd27\" (UID: \"1c463ce3-50ac-473d-bb96-ebc5c3c90e6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dd27"
Oct 31 21:22:46.450541 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:46.450522    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1c463ce3-50ac-473d-bb96-ebc5c3c90e6d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dd27\" (UID: \"1c463ce3-50ac-473d-bb96-ebc5c3c90e6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dd27"
Oct 31 21:22:46.468463 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:46.468439    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7pqj\" (UniqueName: \"kubernetes.io/projected/1c463ce3-50ac-473d-bb96-ebc5c3c90e6d-kube-api-access-k7pqj\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dd27\" (UID: \"1c463ce3-50ac-473d-bb96-ebc5c3c90e6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dd27"
Oct 31 21:22:46.469491 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:46.469470    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c463ce3-50ac-473d-bb96-ebc5c3c90e6d-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-7dd27\" (UID: \"1c463ce3-50ac-473d-bb96-ebc5c3c90e6d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dd27"
Oct 31 21:22:46.618998 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:46.618914    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dd27"
Oct 31 21:22:46.947277 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:46.947237    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dd27"]
Oct 31 21:22:46.949887 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:22:46.949860    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c463ce3_50ac_473d_bb96_ebc5c3c90e6d.slice/crio-b5a46174a4a4b03c91fb08fc061c9e2951f4c355282c3fe61ff60d31c450dbda WatchSource:0}: Error finding container b5a46174a4a4b03c91fb08fc061c9e2951f4c355282c3fe61ff60d31c450dbda: Status 404 returned error can't find the container with id b5a46174a4a4b03c91fb08fc061c9e2951f4c355282c3fe61ff60d31c450dbda
Oct 31 21:22:46.952344 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:46.952316    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:22:46.952419 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:46.952369    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:22:46.958313 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:46.958269    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:47.078285 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:47.078250    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:47.211491 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:47.211406    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:47.350287 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:47.350223    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:47.514956 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:47.514896    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:47.843033 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:47.842922    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:47.900675 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:47.900643    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dd27" event={"ID":"1c463ce3-50ac-473d-bb96-ebc5c3c90e6d","Type":"ContainerStarted","Data":"b6e21037e0f66a1a46b2eeed06c803d12a9a44bce19a4788385b442f5493faa8"}
Oct 31 21:22:47.900822 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:47.900691    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dd27" event={"ID":"1c463ce3-50ac-473d-bb96-ebc5c3c90e6d","Type":"ContainerStarted","Data":"b5a46174a4a4b03c91fb08fc061c9e2951f4c355282c3fe61ff60d31c450dbda"}
Oct 31 21:22:47.900877 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:47.900840    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dd27"
Oct 31 21:22:47.902491 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:47.902471    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dd27"
Oct 31 21:22:47.979239 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:47.979186    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-7dd27" podStartSLOduration=1.979169825 podStartE2EDuration="1.979169825s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:22:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:22:47.964799182 +0000 UTC m=+407.881108613" watchObservedRunningTime="2026-10-31 21:22:47.979169825 +0000 UTC m=+407.895479246"
Oct 31 21:22:48.052425 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:48.052399    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf"]
Oct 31 21:22:48.052643 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:48.052616    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf" podUID="a70a6be8-a62b-4159-bff1-8837e3791e03" containerName="discovery" containerID="cri-o://02f50ecd11292f9fcbfa47b3604435087a01ea769083b6f7302e08c4ad4fbe4f" gracePeriod=30
Oct 31 21:22:48.293371 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:48.293346    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf"
Oct 31 21:22:48.366976 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:48.366888    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a70a6be8-a62b-4159-bff1-8837e3791e03-kube-api-access-dnl48\" (UniqueName: \"kubernetes.io/projected/a70a6be8-a62b-4159-bff1-8837e3791e03-kube-api-access-dnl48\") pod \"a70a6be8-a62b-4159-bff1-8837e3791e03\" (UID: \"a70a6be8-a62b-4159-bff1-8837e3791e03\") "
Oct 31 21:22:48.366976 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:48.366949    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a70a6be8-a62b-4159-bff1-8837e3791e03-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a70a6be8-a62b-4159-bff1-8837e3791e03-istio-csr-dns-cert\") pod \"a70a6be8-a62b-4159-bff1-8837e3791e03\" (UID: \"a70a6be8-a62b-4159-bff1-8837e3791e03\") "
Oct 31 21:22:48.367491 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:48.367111    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a70a6be8-a62b-4159-bff1-8837e3791e03-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a70a6be8-a62b-4159-bff1-8837e3791e03-istio-csr-ca-configmap\") pod \"a70a6be8-a62b-4159-bff1-8837e3791e03\" (UID: \"a70a6be8-a62b-4159-bff1-8837e3791e03\") "
Oct 31 21:22:48.367491 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:48.367148    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a70a6be8-a62b-4159-bff1-8837e3791e03-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a70a6be8-a62b-4159-bff1-8837e3791e03-local-certs\") pod \"a70a6be8-a62b-4159-bff1-8837e3791e03\" (UID: \"a70a6be8-a62b-4159-bff1-8837e3791e03\") "
Oct 31 21:22:48.367491 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:48.367171    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a70a6be8-a62b-4159-bff1-8837e3791e03-istio-token\" (UniqueName: \"kubernetes.io/projected/a70a6be8-a62b-4159-bff1-8837e3791e03-istio-token\") pod \"a70a6be8-a62b-4159-bff1-8837e3791e03\" (UID: \"a70a6be8-a62b-4159-bff1-8837e3791e03\") "
Oct 31 21:22:48.367491 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:48.367200    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a70a6be8-a62b-4159-bff1-8837e3791e03-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a70a6be8-a62b-4159-bff1-8837e3791e03-istio-kubeconfig\") pod \"a70a6be8-a62b-4159-bff1-8837e3791e03\" (UID: \"a70a6be8-a62b-4159-bff1-8837e3791e03\") "
Oct 31 21:22:48.367491 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:48.367255    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a70a6be8-a62b-4159-bff1-8837e3791e03-cacerts\" (UniqueName: \"kubernetes.io/secret/a70a6be8-a62b-4159-bff1-8837e3791e03-cacerts\") pod \"a70a6be8-a62b-4159-bff1-8837e3791e03\" (UID: \"a70a6be8-a62b-4159-bff1-8837e3791e03\") "
Oct 31 21:22:48.367788 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:48.367510    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a70a6be8-a62b-4159-bff1-8837e3791e03-istio-csr-ca-configmap" pod "a70a6be8-a62b-4159-bff1-8837e3791e03" (UID: "a70a6be8-a62b-4159-bff1-8837e3791e03"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:22:48.369451 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:48.369385    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a70a6be8-a62b-4159-bff1-8837e3791e03-istio-csr-dns-cert" pod "a70a6be8-a62b-4159-bff1-8837e3791e03" (UID: "a70a6be8-a62b-4159-bff1-8837e3791e03"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 21:22:48.369575 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:48.369456    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a70a6be8-a62b-4159-bff1-8837e3791e03-kube-api-access-dnl48" pod "a70a6be8-a62b-4159-bff1-8837e3791e03" (UID: "a70a6be8-a62b-4159-bff1-8837e3791e03"). InnerVolumeSpecName "kube-api-access-dnl48". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:22:48.369575 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:48.369486    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a70a6be8-a62b-4159-bff1-8837e3791e03-local-certs" pod "a70a6be8-a62b-4159-bff1-8837e3791e03" (UID: "a70a6be8-a62b-4159-bff1-8837e3791e03"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:22:48.369684 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:48.369626    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a70a6be8-a62b-4159-bff1-8837e3791e03-cacerts" pod "a70a6be8-a62b-4159-bff1-8837e3791e03" (UID: "a70a6be8-a62b-4159-bff1-8837e3791e03"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 21:22:48.369684 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:48.369651    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a70a6be8-a62b-4159-bff1-8837e3791e03-istio-token" pod "a70a6be8-a62b-4159-bff1-8837e3791e03" (UID: "a70a6be8-a62b-4159-bff1-8837e3791e03"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:22:48.369808 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:48.369794    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a70a6be8-a62b-4159-bff1-8837e3791e03-istio-kubeconfig" pod "a70a6be8-a62b-4159-bff1-8837e3791e03" (UID: "a70a6be8-a62b-4159-bff1-8837e3791e03"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 21:22:48.468122 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:48.468090    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dnl48\" (UniqueName: \"kubernetes.io/projected/a70a6be8-a62b-4159-bff1-8837e3791e03-kube-api-access-dnl48\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:22:48.468122 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:48.468117    4460 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a70a6be8-a62b-4159-bff1-8837e3791e03-istio-csr-dns-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:22:48.468122 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:48.468127    4460 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a70a6be8-a62b-4159-bff1-8837e3791e03-istio-csr-ca-configmap\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:22:48.468318 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:48.468136    4460 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a70a6be8-a62b-4159-bff1-8837e3791e03-local-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:22:48.468318 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:48.468147    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a70a6be8-a62b-4159-bff1-8837e3791e03-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:22:48.468318 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:48.468155    4460 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a70a6be8-a62b-4159-bff1-8837e3791e03-istio-kubeconfig\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:22:48.468318 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:48.468162    4460 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a70a6be8-a62b-4159-bff1-8837e3791e03-cacerts\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:22:48.905696 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:48.905660    4460 generic.go:356] "Generic (PLEG): container finished" podID="a70a6be8-a62b-4159-bff1-8837e3791e03" containerID="02f50ecd11292f9fcbfa47b3604435087a01ea769083b6f7302e08c4ad4fbe4f" exitCode=0
Oct 31 21:22:48.905888 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:48.905718    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf"
Oct 31 21:22:48.905888 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:48.905744    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf" event={"ID":"a70a6be8-a62b-4159-bff1-8837e3791e03","Type":"ContainerDied","Data":"02f50ecd11292f9fcbfa47b3604435087a01ea769083b6f7302e08c4ad4fbe4f"}
Oct 31 21:22:48.905888 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:48.905797    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf" event={"ID":"a70a6be8-a62b-4159-bff1-8837e3791e03","Type":"ContainerDied","Data":"df46fcacb035d97faaa6be90710a536c28ab68c05d9212d7a427e812b69b23d4"}
Oct 31 21:22:48.905888 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:48.905818    4460 scope.go:122] "RemoveContainer" containerID="02f50ecd11292f9fcbfa47b3604435087a01ea769083b6f7302e08c4ad4fbe4f"
Oct 31 21:22:48.914297 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:48.914282    4460 scope.go:122] "RemoveContainer" containerID="02f50ecd11292f9fcbfa47b3604435087a01ea769083b6f7302e08c4ad4fbe4f"
Oct 31 21:22:48.914536 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:22:48.914520    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02f50ecd11292f9fcbfa47b3604435087a01ea769083b6f7302e08c4ad4fbe4f\": container with ID starting with 02f50ecd11292f9fcbfa47b3604435087a01ea769083b6f7302e08c4ad4fbe4f not found: ID does not exist" containerID="02f50ecd11292f9fcbfa47b3604435087a01ea769083b6f7302e08c4ad4fbe4f"
Oct 31 21:22:48.914579 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:48.914543    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02f50ecd11292f9fcbfa47b3604435087a01ea769083b6f7302e08c4ad4fbe4f"} err="failed to get container status \"02f50ecd11292f9fcbfa47b3604435087a01ea769083b6f7302e08c4ad4fbe4f\": rpc error: code = NotFound desc = could not find container \"02f50ecd11292f9fcbfa47b3604435087a01ea769083b6f7302e08c4ad4fbe4f\": container with ID starting with 02f50ecd11292f9fcbfa47b3604435087a01ea769083b6f7302e08c4ad4fbe4f not found: ID does not exist"
Oct 31 21:22:48.962244 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:48.962225    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf"]
Oct 31 21:22:48.967761 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:48.967743    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-jkfxf"]
Oct 31 21:22:49.857981 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:49.857950    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:49.972263 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:49.972226    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:50.081162 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:50.081127    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:50.191437 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:50.191359    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:50.223666 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:50.223636    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a70a6be8-a62b-4159-bff1-8837e3791e03" path="/var/lib/kubelet/pods/a70a6be8-a62b-4159-bff1-8837e3791e03/volumes"
Oct 31 21:22:50.303912 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:50.303873    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:51.451158 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:22:51.451122    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:22:52.178515 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:52.178478    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:52.304222 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:52.304182    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:53.756349 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:53.756317    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:53.867651 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:53.867608    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:53.994183 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:53.994145    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:22:56.532545 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:56.532513    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:22:57.908687 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:22:57.908655    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:23:04.149025 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:23:04.148987    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:23:04.262413 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:23:04.262380    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:23:04.659229 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:23:04.659199    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-65b7c64b77-bhgzk"]
Oct 31 21:23:04.659532 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:23:04.659520    4460 cpu_manager.go:414] "containerMap: removing container" podUID="a70a6be8-a62b-4159-bff1-8837e3791e03" containerName="discovery"
Oct 31 21:23:04.659576 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:23:04.659533    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a70a6be8-a62b-4159-bff1-8837e3791e03" containerName="discovery"
Oct 31 21:23:04.659610 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:23:04.659591    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="a70a6be8-a62b-4159-bff1-8837e3791e03" containerName="discovery"
Oct 31 21:23:04.667202 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:23:04.667179    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-65b7c64b77-bhgzk"
Oct 31 21:23:04.684790 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:23:04.684751    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Oct 31 21:23:04.698248 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:23:04.698223    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 21:23:04.698413 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:23:04.698244    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 21:23:04.699754 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:23:04.699735    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-65b7c64b77-bhgzk"]
Oct 31 21:23:04.701300 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:23:04.701279    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8nf8\" (UniqueName: \"kubernetes.io/projected/5d5841bb-6c0c-4771-8742-d7519f0bc582-kube-api-access-r8nf8\") pod \"maas-api-65b7c64b77-bhgzk\" (UID: \"5d5841bb-6c0c-4771-8742-d7519f0bc582\") " pod="odh-ai-gateway-infra/maas-api-65b7c64b77-bhgzk"
Oct 31 21:23:04.701378 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:23:04.701360    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d5841bb-6c0c-4771-8742-d7519f0bc582-metrics-certs\") pod \"maas-api-65b7c64b77-bhgzk\" (UID: \"5d5841bb-6c0c-4771-8742-d7519f0bc582\") " pod="odh-ai-gateway-infra/maas-api-65b7c64b77-bhgzk"
Oct 31 21:23:04.701423 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:23:04.701410    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d5841bb-6c0c-4771-8742-d7519f0bc582-maas-api-tls\") pod \"maas-api-65b7c64b77-bhgzk\" (UID: \"5d5841bb-6c0c-4771-8742-d7519f0bc582\") " pod="odh-ai-gateway-infra/maas-api-65b7c64b77-bhgzk"
Oct 31 21:23:04.701508 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:23:04.701488    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d5841bb-6c0c-4771-8742-d7519f0bc582-openshift-service-ca\") pod \"maas-api-65b7c64b77-bhgzk\" (UID: \"5d5841bb-6c0c-4771-8742-d7519f0bc582\") " pod="odh-ai-gateway-infra/maas-api-65b7c64b77-bhgzk"
Oct 31 21:23:04.785779 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:23:04.785752    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 31 21:23:04.802239 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:23:04.802215    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8nf8\" (UniqueName: \"kubernetes.io/projected/5d5841bb-6c0c-4771-8742-d7519f0bc582-kube-api-access-r8nf8\") pod \"maas-api-65b7c64b77-bhgzk\" (UID: \"5d5841bb-6c0c-4771-8742-d7519f0bc582\") " pod="odh-ai-gateway-infra/maas-api-65b7c64b77-bhgzk"
Oct 31 21:23:04.802352 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:23:04.802247    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d5841bb-6c0c-4771-8742-d7519f0bc582-metrics-certs\") pod \"maas-api-65b7c64b77-bhgzk\" (UID: \"5d5841bb-6c0c-4771-8742-d7519f0bc582\") " pod="odh-ai-gateway-infra/maas-api-65b7c64b77-bhgzk"
Oct 31 21:23:04.802352 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:23:04.802285    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d5841bb-6c0c-4771-8742-d7519f0bc582-maas-api-tls\") pod \"maas-api-65b7c64b77-bhgzk\" (UID: \"5d5841bb-6c0c-4771-8742-d7519f0bc582\") " pod="odh-ai-gateway-infra/maas-api-65b7c64b77-bhgzk"
Oct 31 21:23:04.802352 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:23:04.802321    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d5841bb-6c0c-4771-8742-d7519f0bc582-openshift-service-ca\") pod \"maas-api-65b7c64b77-bhgzk\" (UID: \"5d5841bb-6c0c-4771-8742-d7519f0bc582\") " pod="odh-ai-gateway-infra/maas-api-65b7c64b77-bhgzk"
Oct 31 21:23:04.803002 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:23:04.802976    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d5841bb-6c0c-4771-8742-d7519f0bc582-openshift-service-ca\") pod \"maas-api-65b7c64b77-bhgzk\" (UID: \"5d5841bb-6c0c-4771-8742-d7519f0bc582\") " pod="odh-ai-gateway-infra/maas-api-65b7c64b77-bhgzk"
Oct 31 21:23:04.804889 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:23:04.804863    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d5841bb-6c0c-4771-8742-d7519f0bc582-metrics-certs\") pod \"maas-api-65b7c64b77-bhgzk\" (UID: \"5d5841bb-6c0c-4771-8742-d7519f0bc582\") " pod="odh-ai-gateway-infra/maas-api-65b7c64b77-bhgzk"
Oct 31 21:23:04.804993 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:23:04.804863    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d5841bb-6c0c-4771-8742-d7519f0bc582-maas-api-tls\") pod \"maas-api-65b7c64b77-bhgzk\" (UID: \"5d5841bb-6c0c-4771-8742-d7519f0bc582\") " pod="odh-ai-gateway-infra/maas-api-65b7c64b77-bhgzk"
Oct 31 21:23:04.813181 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:23:04.813164    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-kmnpr\""
Oct 31 21:23:04.828835 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:23:04.828813    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8nf8\" (UniqueName: \"kubernetes.io/projected/5d5841bb-6c0c-4771-8742-d7519f0bc582-kube-api-access-r8nf8\") pod \"maas-api-65b7c64b77-bhgzk\" (UID: \"5d5841bb-6c0c-4771-8742-d7519f0bc582\") " pod="odh-ai-gateway-infra/maas-api-65b7c64b77-bhgzk"
Oct 31 21:23:04.986086 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:23:04.986008    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-65b7c64b77-bhgzk"
Oct 31 21:23:05.296631 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:23:05.296521    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-65b7c64b77-bhgzk"]
Oct 31 21:23:05.299296 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:23:05.299269    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d5841bb_6c0c_4771_8742_d7519f0bc582.slice/crio-9a38668d5c3195b948ebe4e78c7cbe860a75a5c72b9b4a61275251538309b574 WatchSource:0}: Error finding container 9a38668d5c3195b948ebe4e78c7cbe860a75a5c72b9b4a61275251538309b574: Status 404 returned error can't find the container with id 9a38668d5c3195b948ebe4e78c7cbe860a75a5c72b9b4a61275251538309b574
Oct 31 21:23:05.978858 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:23:05.978816    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-65b7c64b77-bhgzk" event={"ID":"5d5841bb-6c0c-4771-8742-d7519f0bc582","Type":"ContainerStarted","Data":"9a38668d5c3195b948ebe4e78c7cbe860a75a5c72b9b4a61275251538309b574"}
Oct 31 21:23:07.989484 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:23:07.989441    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-65b7c64b77-bhgzk" event={"ID":"5d5841bb-6c0c-4771-8742-d7519f0bc582","Type":"ContainerStarted","Data":"40297d97fc8f7a5c8458cd4d46f43e5d0fa8f502fe57d43c0fa4fee0925320a9"}
Oct 31 21:23:07.989872 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:23:07.989561    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-65b7c64b77-bhgzk"
Oct 31 21:23:08.036159 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:23:08.036118    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-65b7c64b77-bhgzk" podStartSLOduration=1.942166882 podStartE2EDuration="4.03610454s" totalImagesPullingTime="2.093937658s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:23:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:23:08.035392028 +0000 UTC m=+427.951701458" watchObservedRunningTime="2026-10-31 21:23:08.03610454 +0000 UTC m=+427.952413968"
Oct 31 21:23:13.998869 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:23:13.998838    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-65b7c64b77-bhgzk"
Oct 31 21:23:21.450744 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:23:21.450704    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:23:43.743816 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:23:43.743766    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:23:51.450607 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:23:51.450565    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:24:21.450072 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:24:21.449997    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:24:22.110370 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:22.110333    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:24:22.126744 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:22.126713    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-lsn4g"]
Oct 31 21:24:22.130028 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:22.130011    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d85675f98-lsn4g"
Oct 31 21:24:22.153753 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:22.153723    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-lsn4g"]
Oct 31 21:24:22.163354 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:22.163333    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\""
Oct 31 21:24:22.163354 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:22.163344    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-vjx9r\""
Oct 31 21:24:22.163513 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:22.163375    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 31 21:24:22.163513 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:22.163418    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 31 21:24:22.163513 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:22.163421    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Oct 31 21:24:22.200283 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:22.200255    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e82ef8c-c696-42bf-9e22-142e83bec0c6-metrics-certs\") pod \"ai-gateway-controller-7d85675f98-lsn4g\" (UID: \"5e82ef8c-c696-42bf-9e22-142e83bec0c6\") " pod="opendatahub/ai-gateway-controller-7d85675f98-lsn4g"
Oct 31 21:24:22.200385 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:22.200314    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2w22\" (UniqueName: \"kubernetes.io/projected/5e82ef8c-c696-42bf-9e22-142e83bec0c6-kube-api-access-l2w22\") pod \"ai-gateway-controller-7d85675f98-lsn4g\" (UID: \"5e82ef8c-c696-42bf-9e22-142e83bec0c6\") " pod="opendatahub/ai-gateway-controller-7d85675f98-lsn4g"
Oct 31 21:24:22.300927 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:22.300894    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2w22\" (UniqueName: \"kubernetes.io/projected/5e82ef8c-c696-42bf-9e22-142e83bec0c6-kube-api-access-l2w22\") pod \"ai-gateway-controller-7d85675f98-lsn4g\" (UID: \"5e82ef8c-c696-42bf-9e22-142e83bec0c6\") " pod="opendatahub/ai-gateway-controller-7d85675f98-lsn4g"
Oct 31 21:24:22.301071 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:22.300993    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e82ef8c-c696-42bf-9e22-142e83bec0c6-metrics-certs\") pod \"ai-gateway-controller-7d85675f98-lsn4g\" (UID: \"5e82ef8c-c696-42bf-9e22-142e83bec0c6\") " pod="opendatahub/ai-gateway-controller-7d85675f98-lsn4g"
Oct 31 21:24:22.303350 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:22.303326    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e82ef8c-c696-42bf-9e22-142e83bec0c6-metrics-certs\") pod \"ai-gateway-controller-7d85675f98-lsn4g\" (UID: \"5e82ef8c-c696-42bf-9e22-142e83bec0c6\") " pod="opendatahub/ai-gateway-controller-7d85675f98-lsn4g"
Oct 31 21:24:22.310327 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:22.310304    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:24:22.325999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:22.325978    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2w22\" (UniqueName: \"kubernetes.io/projected/5e82ef8c-c696-42bf-9e22-142e83bec0c6-kube-api-access-l2w22\") pod \"ai-gateway-controller-7d85675f98-lsn4g\" (UID: \"5e82ef8c-c696-42bf-9e22-142e83bec0c6\") " pod="opendatahub/ai-gateway-controller-7d85675f98-lsn4g"
Oct 31 21:24:22.442341 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:22.442263    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d85675f98-lsn4g"
Oct 31 21:24:22.743211 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:22.742252    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-lsn4g"]
Oct 31 21:24:22.745447 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:24:22.745420    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e82ef8c_c696_42bf_9e22_142e83bec0c6.slice/crio-e8a62de9bfc2e7082052d876e1e4609b889048fb85fb854e9087f4d625b77b18 WatchSource:0}: Error finding container e8a62de9bfc2e7082052d876e1e4609b889048fb85fb854e9087f4d625b77b18: Status 404 returned error can't find the container with id e8a62de9bfc2e7082052d876e1e4609b889048fb85fb854e9087f4d625b77b18
Oct 31 21:24:23.291742 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:23.291699    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d85675f98-lsn4g" event={"ID":"5e82ef8c-c696-42bf-9e22-142e83bec0c6","Type":"ContainerStarted","Data":"e8a62de9bfc2e7082052d876e1e4609b889048fb85fb854e9087f4d625b77b18"}
Oct 31 21:24:24.302601 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:24.302564    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d85675f98-lsn4g" event={"ID":"5e82ef8c-c696-42bf-9e22-142e83bec0c6","Type":"ContainerStarted","Data":"47b9170b56642ee3b0bf51a43ee5c95080b8c831ae1a8250854a0282a2755c40"}
Oct 31 21:24:24.303032 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:24.302678    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-7d85675f98-lsn4g"
Oct 31 21:24:24.347289 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:24.347242    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-7d85675f98-lsn4g" podStartSLOduration=0.918883394 podStartE2EDuration="2.347230772s" totalImagesPullingTime="1.428347378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:24:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:24:24.346936053 +0000 UTC m=+504.263257718" watchObservedRunningTime="2026-10-31 21:24:24.347230772 +0000 UTC m=+504.263540200"
Oct 31 21:24:26.288053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:26.288011    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:24:26.403543 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:26.403514    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-55fbbbf9dc-hxh82"]
Oct 31 21:24:26.406984 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:26.406967    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-55fbbbf9dc-hxh82"
Oct 31 21:24:26.423634 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:26.423615    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-627x2\""
Oct 31 21:24:26.429995 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:26.429968    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:24:26.437468 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:26.437445    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Oct 31 21:24:26.439011 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:26.438991    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-55fbbbf9dc-hxh82"]
Oct 31 21:24:26.538160 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:26.538068    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwx8z\" (UniqueName: \"kubernetes.io/projected/bb3b6872-a3ff-438d-b1ba-45fca4844b6e-kube-api-access-kwx8z\") pod \"payload-processing-55fbbbf9dc-hxh82\" (UID: \"bb3b6872-a3ff-438d-b1ba-45fca4844b6e\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-hxh82"
Oct 31 21:24:26.538160 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:26.538119    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb3b6872-a3ff-438d-b1ba-45fca4844b6e-plugins-config-volume\") pod \"payload-processing-55fbbbf9dc-hxh82\" (UID: \"bb3b6872-a3ff-438d-b1ba-45fca4844b6e\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-hxh82"
Oct 31 21:24:26.638773 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:26.638742    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwx8z\" (UniqueName: \"kubernetes.io/projected/bb3b6872-a3ff-438d-b1ba-45fca4844b6e-kube-api-access-kwx8z\") pod \"payload-processing-55fbbbf9dc-hxh82\" (UID: \"bb3b6872-a3ff-438d-b1ba-45fca4844b6e\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-hxh82"
Oct 31 21:24:26.638923 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:26.638782    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb3b6872-a3ff-438d-b1ba-45fca4844b6e-plugins-config-volume\") pod \"payload-processing-55fbbbf9dc-hxh82\" (UID: \"bb3b6872-a3ff-438d-b1ba-45fca4844b6e\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-hxh82"
Oct 31 21:24:26.639572 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:26.639554    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb3b6872-a3ff-438d-b1ba-45fca4844b6e-plugins-config-volume\") pod \"payload-processing-55fbbbf9dc-hxh82\" (UID: \"bb3b6872-a3ff-438d-b1ba-45fca4844b6e\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-hxh82"
Oct 31 21:24:26.663834 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:26.663800    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwx8z\" (UniqueName: \"kubernetes.io/projected/bb3b6872-a3ff-438d-b1ba-45fca4844b6e-kube-api-access-kwx8z\") pod \"payload-processing-55fbbbf9dc-hxh82\" (UID: \"bb3b6872-a3ff-438d-b1ba-45fca4844b6e\") " pod="openshift-ingress/payload-processing-55fbbbf9dc-hxh82"
Oct 31 21:24:26.719378 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:26.719348    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-55fbbbf9dc-hxh82"
Oct 31 21:24:27.015216 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:27.015193    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-55fbbbf9dc-hxh82"]
Oct 31 21:24:27.017326 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:24:27.017292    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb3b6872_a3ff_438d_b1ba_45fca4844b6e.slice/crio-0dff23a96141d040a0ae4ab47b9700c72b95afafb72fb80ef4bb7f74b4107a06 WatchSource:0}: Error finding container 0dff23a96141d040a0ae4ab47b9700c72b95afafb72fb80ef4bb7f74b4107a06: Status 404 returned error can't find the container with id 0dff23a96141d040a0ae4ab47b9700c72b95afafb72fb80ef4bb7f74b4107a06
Oct 31 21:24:27.317802 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:27.317715    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-55fbbbf9dc-hxh82" event={"ID":"bb3b6872-a3ff-438d-b1ba-45fca4844b6e","Type":"ContainerStarted","Data":"0dff23a96141d040a0ae4ab47b9700c72b95afafb72fb80ef4bb7f74b4107a06"}
Oct 31 21:24:29.327765 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:29.327682    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-55fbbbf9dc-hxh82" event={"ID":"bb3b6872-a3ff-438d-b1ba-45fca4844b6e","Type":"ContainerStarted","Data":"c9a3347331dad8775b12a898aaff6ae8519438e7c888f9575c7903d828e133f2"}
Oct 31 21:24:29.328139 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:29.327780    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-55fbbbf9dc-hxh82"
Oct 31 21:24:29.373584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:29.373536    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-55fbbbf9dc-hxh82" podStartSLOduration=1.326890104 podStartE2EDuration="3.373523276s" totalImagesPullingTime="2.046633172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:24:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:24:29.373000237 +0000 UTC m=+509.289309668" watchObservedRunningTime="2026-10-31 21:24:29.373523276 +0000 UTC m=+509.289832702"
Oct 31 21:24:35.310512 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:35.310478    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-7d85675f98-lsn4g"
Oct 31 21:24:35.603330 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:35.603247    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-6f5dcfbf7c-4gxsx"]
Oct 31 21:24:35.606909 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:35.606886    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-6f5dcfbf7c-4gxsx"
Oct 31 21:24:35.628093 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:35.628066    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-6f5dcfbf7c-4gxsx"]
Oct 31 21:24:35.713594 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:35.713562    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2ac0597-8657-4e9f-9585-fdac218122b3-metrics-certs\") pod \"ai-gateway-controller-6f5dcfbf7c-4gxsx\" (UID: \"f2ac0597-8657-4e9f-9585-fdac218122b3\") " pod="opendatahub/ai-gateway-controller-6f5dcfbf7c-4gxsx"
Oct 31 21:24:35.713594 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:35.713599    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44t94\" (UniqueName: \"kubernetes.io/projected/f2ac0597-8657-4e9f-9585-fdac218122b3-kube-api-access-44t94\") pod \"ai-gateway-controller-6f5dcfbf7c-4gxsx\" (UID: \"f2ac0597-8657-4e9f-9585-fdac218122b3\") " pod="opendatahub/ai-gateway-controller-6f5dcfbf7c-4gxsx"
Oct 31 21:24:35.814677 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:35.814648    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2ac0597-8657-4e9f-9585-fdac218122b3-metrics-certs\") pod \"ai-gateway-controller-6f5dcfbf7c-4gxsx\" (UID: \"f2ac0597-8657-4e9f-9585-fdac218122b3\") " pod="opendatahub/ai-gateway-controller-6f5dcfbf7c-4gxsx"
Oct 31 21:24:35.814677 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:35.814679    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44t94\" (UniqueName: \"kubernetes.io/projected/f2ac0597-8657-4e9f-9585-fdac218122b3-kube-api-access-44t94\") pod \"ai-gateway-controller-6f5dcfbf7c-4gxsx\" (UID: \"f2ac0597-8657-4e9f-9585-fdac218122b3\") " pod="opendatahub/ai-gateway-controller-6f5dcfbf7c-4gxsx"
Oct 31 21:24:35.816956 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:35.816919    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2ac0597-8657-4e9f-9585-fdac218122b3-metrics-certs\") pod \"ai-gateway-controller-6f5dcfbf7c-4gxsx\" (UID: \"f2ac0597-8657-4e9f-9585-fdac218122b3\") " pod="opendatahub/ai-gateway-controller-6f5dcfbf7c-4gxsx"
Oct 31 21:24:35.839867 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:35.839839    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44t94\" (UniqueName: \"kubernetes.io/projected/f2ac0597-8657-4e9f-9585-fdac218122b3-kube-api-access-44t94\") pod \"ai-gateway-controller-6f5dcfbf7c-4gxsx\" (UID: \"f2ac0597-8657-4e9f-9585-fdac218122b3\") " pod="opendatahub/ai-gateway-controller-6f5dcfbf7c-4gxsx"
Oct 31 21:24:35.919617 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:35.919546    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-6f5dcfbf7c-4gxsx"
Oct 31 21:24:36.217584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:36.217561    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-6f5dcfbf7c-4gxsx"]
Oct 31 21:24:36.219477 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:24:36.219449    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2ac0597_8657_4e9f_9585_fdac218122b3.slice/crio-b363d099d3d432d2c49980c448ab44518c0d135efe70d39db45a43f3864355ef WatchSource:0}: Error finding container b363d099d3d432d2c49980c448ab44518c0d135efe70d39db45a43f3864355ef: Status 404 returned error can't find the container with id b363d099d3d432d2c49980c448ab44518c0d135efe70d39db45a43f3864355ef
Oct 31 21:24:36.362856 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:36.362824    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-6f5dcfbf7c-4gxsx" event={"ID":"f2ac0597-8657-4e9f-9585-fdac218122b3","Type":"ContainerStarted","Data":"b363d099d3d432d2c49980c448ab44518c0d135efe70d39db45a43f3864355ef"}
Oct 31 21:24:37.371358 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:37.371320    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-6f5dcfbf7c-4gxsx" event={"ID":"f2ac0597-8657-4e9f-9585-fdac218122b3","Type":"ContainerStarted","Data":"5d98318e91611263d5a3a5c2b01b124033e82bfa53d43d140298b8d4fc5ba0a6"}
Oct 31 21:24:37.371730 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:37.371439    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-6f5dcfbf7c-4gxsx"
Oct 31 21:24:37.415946 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:37.415903    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-6f5dcfbf7c-4gxsx" podStartSLOduration=2.024632435 podStartE2EDuration="2.415891086s" totalImagesPullingTime="391.258651ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:24:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:24:37.415059201 +0000 UTC m=+517.331368643" watchObservedRunningTime="2026-10-31 21:24:37.415891086 +0000 UTC m=+517.332200514"
Oct 31 21:24:40.334396 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:40.334367    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-55fbbbf9dc-hxh82"
Oct 31 21:24:43.857565 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:43.857528    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:24:48.377840 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:48.377806    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-6f5dcfbf7c-4gxsx"
Oct 31 21:24:48.451906 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:48.451870    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-lsn4g"]
Oct 31 21:24:48.452174 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:48.452148    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-7d85675f98-lsn4g" podUID="5e82ef8c-c696-42bf-9e22-142e83bec0c6" containerName="manager" containerID="cri-o://47b9170b56642ee3b0bf51a43ee5c95080b8c831ae1a8250854a0282a2755c40" gracePeriod=30
Oct 31 21:24:48.690577 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:48.690554    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d85675f98-lsn4g"
Oct 31 21:24:48.824386 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:48.824355    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e82ef8c-c696-42bf-9e22-142e83bec0c6-kube-api-access-l2w22\" (UniqueName: \"kubernetes.io/projected/5e82ef8c-c696-42bf-9e22-142e83bec0c6-kube-api-access-l2w22\") pod \"5e82ef8c-c696-42bf-9e22-142e83bec0c6\" (UID: \"5e82ef8c-c696-42bf-9e22-142e83bec0c6\") "
Oct 31 21:24:48.824549 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:48.824398    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5e82ef8c-c696-42bf-9e22-142e83bec0c6-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e82ef8c-c696-42bf-9e22-142e83bec0c6-metrics-certs\") pod \"5e82ef8c-c696-42bf-9e22-142e83bec0c6\" (UID: \"5e82ef8c-c696-42bf-9e22-142e83bec0c6\") "
Oct 31 21:24:48.826469 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:48.826437    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e82ef8c-c696-42bf-9e22-142e83bec0c6-metrics-certs" pod "5e82ef8c-c696-42bf-9e22-142e83bec0c6" (UID: "5e82ef8c-c696-42bf-9e22-142e83bec0c6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 21:24:48.826606 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:48.826501    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e82ef8c-c696-42bf-9e22-142e83bec0c6-kube-api-access-l2w22" pod "5e82ef8c-c696-42bf-9e22-142e83bec0c6" (UID: "5e82ef8c-c696-42bf-9e22-142e83bec0c6"). InnerVolumeSpecName "kube-api-access-l2w22". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:24:48.925774 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:48.925702    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2w22\" (UniqueName: \"kubernetes.io/projected/5e82ef8c-c696-42bf-9e22-142e83bec0c6-kube-api-access-l2w22\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:24:48.925774 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:48.925729    4460 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e82ef8c-c696-42bf-9e22-142e83bec0c6-metrics-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:24:49.422404 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:49.422370    4460 generic.go:356] "Generic (PLEG): container finished" podID="5e82ef8c-c696-42bf-9e22-142e83bec0c6" containerID="47b9170b56642ee3b0bf51a43ee5c95080b8c831ae1a8250854a0282a2755c40" exitCode=0
Oct 31 21:24:49.422814 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:49.422415    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d85675f98-lsn4g" event={"ID":"5e82ef8c-c696-42bf-9e22-142e83bec0c6","Type":"ContainerDied","Data":"47b9170b56642ee3b0bf51a43ee5c95080b8c831ae1a8250854a0282a2755c40"}
Oct 31 21:24:49.422814 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:49.422426    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d85675f98-lsn4g"
Oct 31 21:24:49.422814 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:49.422480    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d85675f98-lsn4g" event={"ID":"5e82ef8c-c696-42bf-9e22-142e83bec0c6","Type":"ContainerDied","Data":"e8a62de9bfc2e7082052d876e1e4609b889048fb85fb854e9087f4d625b77b18"}
Oct 31 21:24:49.422814 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:49.422502    4460 scope.go:122] "RemoveContainer" containerID="47b9170b56642ee3b0bf51a43ee5c95080b8c831ae1a8250854a0282a2755c40"
Oct 31 21:24:49.430565 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:49.430549    4460 scope.go:122] "RemoveContainer" containerID="47b9170b56642ee3b0bf51a43ee5c95080b8c831ae1a8250854a0282a2755c40"
Oct 31 21:24:49.430836 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:24:49.430819    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47b9170b56642ee3b0bf51a43ee5c95080b8c831ae1a8250854a0282a2755c40\": container with ID starting with 47b9170b56642ee3b0bf51a43ee5c95080b8c831ae1a8250854a0282a2755c40 not found: ID does not exist" containerID="47b9170b56642ee3b0bf51a43ee5c95080b8c831ae1a8250854a0282a2755c40"
Oct 31 21:24:49.430883 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:49.430844    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47b9170b56642ee3b0bf51a43ee5c95080b8c831ae1a8250854a0282a2755c40"} err="failed to get container status \"47b9170b56642ee3b0bf51a43ee5c95080b8c831ae1a8250854a0282a2755c40\": rpc error: code = NotFound desc = could not find container \"47b9170b56642ee3b0bf51a43ee5c95080b8c831ae1a8250854a0282a2755c40\": container with ID starting with 47b9170b56642ee3b0bf51a43ee5c95080b8c831ae1a8250854a0282a2755c40 not found: ID does not exist"
Oct 31 21:24:49.480161 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:49.480132    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-lsn4g"]
Oct 31 21:24:49.493692 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:49.493673    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-lsn4g"]
Oct 31 21:24:50.095655 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:50.095622    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:24:50.223299 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:50.223269    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5e82ef8c-c696-42bf-9e22-142e83bec0c6" path="/var/lib/kubelet/pods/5e82ef8c-c696-42bf-9e22-142e83bec0c6/volumes"
Oct 31 21:24:50.601211 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:50.601175    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:24:51.455393 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:24:51.455352    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:24:51.855655 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:51.855620    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:24:51.969723 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:24:51.969692    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:25:08.796682 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:08.796645    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:25:09.863650 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:09.863618    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:25:12.415361 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:12.415327    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-bfqc5"]
Oct 31 21:25:12.415821 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:12.415677    4460 cpu_manager.go:414] "containerMap: removing container" podUID="5e82ef8c-c696-42bf-9e22-142e83bec0c6" containerName="manager"
Oct 31 21:25:12.415821 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:12.415687    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5e82ef8c-c696-42bf-9e22-142e83bec0c6" containerName="manager"
Oct 31 21:25:12.415821 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:12.415746    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="5e82ef8c-c696-42bf-9e22-142e83bec0c6" containerName="manager"
Oct 31 21:25:12.423211 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:12.423188    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bfqc5"
Oct 31 21:25:12.442966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:12.442919    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 31 21:25:12.453290 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:12.453262    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 31 21:25:12.454727 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:12.454707    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-bfqc5"]
Oct 31 21:25:12.456781 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:12.456765    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-r4s2n\""
Oct 31 21:25:12.456911 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:12.456898    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Oct 31 21:25:12.483824 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:12.483793    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:25:12.519895 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:12.519869    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/678a541a-87a5-4280-b4d4-24ba4f13df8e-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bfqc5\" (UID: \"678a541a-87a5-4280-b4d4-24ba4f13df8e\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bfqc5"
Oct 31 21:25:12.520105 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:12.519921    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/678a541a-87a5-4280-b4d4-24ba4f13df8e-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bfqc5\" (UID: \"678a541a-87a5-4280-b4d4-24ba4f13df8e\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bfqc5"
Oct 31 21:25:12.520105 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:12.520023    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/678a541a-87a5-4280-b4d4-24ba4f13df8e-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bfqc5\" (UID: \"678a541a-87a5-4280-b4d4-24ba4f13df8e\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bfqc5"
Oct 31 21:25:12.520220 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:12.520101    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/678a541a-87a5-4280-b4d4-24ba4f13df8e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bfqc5\" (UID: \"678a541a-87a5-4280-b4d4-24ba4f13df8e\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bfqc5"
Oct 31 21:25:12.520220 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:12.520163    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/678a541a-87a5-4280-b4d4-24ba4f13df8e-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bfqc5\" (UID: \"678a541a-87a5-4280-b4d4-24ba4f13df8e\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bfqc5"
Oct 31 21:25:12.520283 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:12.520248    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtdtp\" (UniqueName: \"kubernetes.io/projected/678a541a-87a5-4280-b4d4-24ba4f13df8e-kube-api-access-xtdtp\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bfqc5\" (UID: \"678a541a-87a5-4280-b4d4-24ba4f13df8e\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bfqc5"
Oct 31 21:25:12.621312 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:12.621269    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/678a541a-87a5-4280-b4d4-24ba4f13df8e-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bfqc5\" (UID: \"678a541a-87a5-4280-b4d4-24ba4f13df8e\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bfqc5"
Oct 31 21:25:12.621508 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:12.621328    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/678a541a-87a5-4280-b4d4-24ba4f13df8e-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bfqc5\" (UID: \"678a541a-87a5-4280-b4d4-24ba4f13df8e\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bfqc5"
Oct 31 21:25:12.621508 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:12.621354    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/678a541a-87a5-4280-b4d4-24ba4f13df8e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bfqc5\" (UID: \"678a541a-87a5-4280-b4d4-24ba4f13df8e\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bfqc5"
Oct 31 21:25:12.621508 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:12.621388    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/678a541a-87a5-4280-b4d4-24ba4f13df8e-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bfqc5\" (UID: \"678a541a-87a5-4280-b4d4-24ba4f13df8e\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bfqc5"
Oct 31 21:25:12.621508 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:12.621439    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtdtp\" (UniqueName: \"kubernetes.io/projected/678a541a-87a5-4280-b4d4-24ba4f13df8e-kube-api-access-xtdtp\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bfqc5\" (UID: \"678a541a-87a5-4280-b4d4-24ba4f13df8e\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bfqc5"
Oct 31 21:25:12.621508 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:12.621464    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/678a541a-87a5-4280-b4d4-24ba4f13df8e-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bfqc5\" (UID: \"678a541a-87a5-4280-b4d4-24ba4f13df8e\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bfqc5"
Oct 31 21:25:12.621856 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:12.621823    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/678a541a-87a5-4280-b4d4-24ba4f13df8e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bfqc5\" (UID: \"678a541a-87a5-4280-b4d4-24ba4f13df8e\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bfqc5"
Oct 31 21:25:12.621931 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:12.621865    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/678a541a-87a5-4280-b4d4-24ba4f13df8e-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bfqc5\" (UID: \"678a541a-87a5-4280-b4d4-24ba4f13df8e\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bfqc5"
Oct 31 21:25:12.621931 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:12.621907    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/678a541a-87a5-4280-b4d4-24ba4f13df8e-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bfqc5\" (UID: \"678a541a-87a5-4280-b4d4-24ba4f13df8e\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bfqc5"
Oct 31 21:25:12.623703 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:12.623681    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/678a541a-87a5-4280-b4d4-24ba4f13df8e-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bfqc5\" (UID: \"678a541a-87a5-4280-b4d4-24ba4f13df8e\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bfqc5"
Oct 31 21:25:12.623868 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:12.623851    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/678a541a-87a5-4280-b4d4-24ba4f13df8e-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bfqc5\" (UID: \"678a541a-87a5-4280-b4d4-24ba4f13df8e\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bfqc5"
Oct 31 21:25:12.649741 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:12.649712    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtdtp\" (UniqueName: \"kubernetes.io/projected/678a541a-87a5-4280-b4d4-24ba4f13df8e-kube-api-access-xtdtp\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-bfqc5\" (UID: \"678a541a-87a5-4280-b4d4-24ba4f13df8e\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bfqc5"
Oct 31 21:25:12.737185 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:12.737105    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bfqc5"
Oct 31 21:25:13.047397 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:13.047371    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-bfqc5"]
Oct 31 21:25:13.049394 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:25:13.049367    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod678a541a_87a5_4280_b4d4_24ba4f13df8e.slice/crio-55e47c1e6e76c52fd10d1990229aa937c5ffb9325e2c6c2be0e693302ee39e4a WatchSource:0}: Error finding container 55e47c1e6e76c52fd10d1990229aa937c5ffb9325e2c6c2be0e693302ee39e4a: Status 404 returned error can't find the container with id 55e47c1e6e76c52fd10d1990229aa937c5ffb9325e2c6c2be0e693302ee39e4a
Oct 31 21:25:13.529727 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:13.529687    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bfqc5" event={"ID":"678a541a-87a5-4280-b4d4-24ba4f13df8e","Type":"ContainerStarted","Data":"55e47c1e6e76c52fd10d1990229aa937c5ffb9325e2c6c2be0e693302ee39e4a"}
Oct 31 21:25:13.801978 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:13.801869    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-bnv8m"]
Oct 31 21:25:13.806005 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:13.805451    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bnv8m"
Oct 31 21:25:13.822423 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:13.822392    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Oct 31 21:25:13.837086 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:13.837052    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-bnv8m"]
Oct 31 21:25:13.863245 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:13.863193    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:25:13.931365 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:13.931326    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fshsq\" (UniqueName: \"kubernetes.io/projected/df78aa1c-2468-44bb-a844-6d02c9b319f3-kube-api-access-fshsq\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bnv8m\" (UID: \"df78aa1c-2468-44bb-a844-6d02c9b319f3\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bnv8m"
Oct 31 21:25:13.931555 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:13.931384    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df78aa1c-2468-44bb-a844-6d02c9b319f3-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bnv8m\" (UID: \"df78aa1c-2468-44bb-a844-6d02c9b319f3\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bnv8m"
Oct 31 21:25:13.931555 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:13.931510    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df78aa1c-2468-44bb-a844-6d02c9b319f3-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bnv8m\" (UID: \"df78aa1c-2468-44bb-a844-6d02c9b319f3\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bnv8m"
Oct 31 21:25:13.931555 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:13.931539    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df78aa1c-2468-44bb-a844-6d02c9b319f3-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bnv8m\" (UID: \"df78aa1c-2468-44bb-a844-6d02c9b319f3\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bnv8m"
Oct 31 21:25:13.931699 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:13.931577    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df78aa1c-2468-44bb-a844-6d02c9b319f3-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bnv8m\" (UID: \"df78aa1c-2468-44bb-a844-6d02c9b319f3\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bnv8m"
Oct 31 21:25:13.931765 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:13.931731    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df78aa1c-2468-44bb-a844-6d02c9b319f3-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bnv8m\" (UID: \"df78aa1c-2468-44bb-a844-6d02c9b319f3\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bnv8m"
Oct 31 21:25:14.032534 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:14.032458    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df78aa1c-2468-44bb-a844-6d02c9b319f3-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bnv8m\" (UID: \"df78aa1c-2468-44bb-a844-6d02c9b319f3\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bnv8m"
Oct 31 21:25:14.032534 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:14.032504    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fshsq\" (UniqueName: \"kubernetes.io/projected/df78aa1c-2468-44bb-a844-6d02c9b319f3-kube-api-access-fshsq\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bnv8m\" (UID: \"df78aa1c-2468-44bb-a844-6d02c9b319f3\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bnv8m"
Oct 31 21:25:14.032618 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:14.032591    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df78aa1c-2468-44bb-a844-6d02c9b319f3-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bnv8m\" (UID: \"df78aa1c-2468-44bb-a844-6d02c9b319f3\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bnv8m"
Oct 31 21:25:14.032744 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:14.032624    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df78aa1c-2468-44bb-a844-6d02c9b319f3-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bnv8m\" (UID: \"df78aa1c-2468-44bb-a844-6d02c9b319f3\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bnv8m"
Oct 31 21:25:14.032744 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:14.032660    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df78aa1c-2468-44bb-a844-6d02c9b319f3-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bnv8m\" (UID: \"df78aa1c-2468-44bb-a844-6d02c9b319f3\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bnv8m"
Oct 31 21:25:14.032744 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:14.032696    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df78aa1c-2468-44bb-a844-6d02c9b319f3-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bnv8m\" (UID: \"df78aa1c-2468-44bb-a844-6d02c9b319f3\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bnv8m"
Oct 31 21:25:14.032885 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:14.032834    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df78aa1c-2468-44bb-a844-6d02c9b319f3-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bnv8m\" (UID: \"df78aa1c-2468-44bb-a844-6d02c9b319f3\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bnv8m"
Oct 31 21:25:14.033042 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:14.033020    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df78aa1c-2468-44bb-a844-6d02c9b319f3-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bnv8m\" (UID: \"df78aa1c-2468-44bb-a844-6d02c9b319f3\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bnv8m"
Oct 31 21:25:14.033110 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:14.033042    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df78aa1c-2468-44bb-a844-6d02c9b319f3-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bnv8m\" (UID: \"df78aa1c-2468-44bb-a844-6d02c9b319f3\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bnv8m"
Oct 31 21:25:14.034859 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:14.034831    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df78aa1c-2468-44bb-a844-6d02c9b319f3-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bnv8m\" (UID: \"df78aa1c-2468-44bb-a844-6d02c9b319f3\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bnv8m"
Oct 31 21:25:14.034960 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:14.034925    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df78aa1c-2468-44bb-a844-6d02c9b319f3-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bnv8m\" (UID: \"df78aa1c-2468-44bb-a844-6d02c9b319f3\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bnv8m"
Oct 31 21:25:14.057395 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:14.057349    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fshsq\" (UniqueName: \"kubernetes.io/projected/df78aa1c-2468-44bb-a844-6d02c9b319f3-kube-api-access-fshsq\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-bnv8m\" (UID: \"df78aa1c-2468-44bb-a844-6d02c9b319f3\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bnv8m"
Oct 31 21:25:14.122351 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:14.122324    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bnv8m"
Oct 31 21:25:14.426132 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:14.425957    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-bnv8m"]
Oct 31 21:25:14.429059 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:25:14.429022    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf78aa1c_2468_44bb_a844_6d02c9b319f3.slice/crio-5d3b671b21d22ee63d85ec0b987922ff59be3e538ca1c01845218e08c05a72c4 WatchSource:0}: Error finding container 5d3b671b21d22ee63d85ec0b987922ff59be3e538ca1c01845218e08c05a72c4: Status 404 returned error can't find the container with id 5d3b671b21d22ee63d85ec0b987922ff59be3e538ca1c01845218e08c05a72c4
Oct 31 21:25:14.535362 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:14.535321    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bfqc5" event={"ID":"678a541a-87a5-4280-b4d4-24ba4f13df8e","Type":"ContainerStarted","Data":"f02178145cefef434387cfda39b3e1743b716a2940a8a48df7acf25e0cc51083"}
Oct 31 21:25:14.536563 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:14.536536    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bnv8m" event={"ID":"df78aa1c-2468-44bb-a844-6d02c9b319f3","Type":"ContainerStarted","Data":"5d3b671b21d22ee63d85ec0b987922ff59be3e538ca1c01845218e08c05a72c4"}
Oct 31 21:25:14.583661 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:14.583607    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bfqc5" podStartSLOduration=1.604410925 podStartE2EDuration="2.583590738s" totalImagesPullingTime="979.179813ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:25:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:25:14.582399902 +0000 UTC m=+554.498709332" watchObservedRunningTime="2026-10-31 21:25:14.583590738 +0000 UTC m=+554.499900166"
Oct 31 21:25:14.738322 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:14.738239    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bfqc5"
Oct 31 21:25:14.796148 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:14.796118    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:25:14.866421 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:14.866394    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:25:15.546598 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:15.546558    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bnv8m" event={"ID":"df78aa1c-2468-44bb-a844-6d02c9b319f3","Type":"ContainerStarted","Data":"e7a6bfa6ff50c92b52e14ceba17baf577335951c4e5988a9bfc8a4e98b95caaf"}
Oct 31 21:25:15.602554 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:15.602487    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bnv8m" podStartSLOduration=2.430822565 podStartE2EDuration="2.602472919s" totalImagesPullingTime="171.650354ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:25:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:25:15.60219079 +0000 UTC m=+555.518500227" watchObservedRunningTime="2026-10-31 21:25:15.602472919 +0000 UTC m=+555.518782397"
Oct 31 21:25:15.644007 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:15.643975    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-xcmm7"]
Oct 31 21:25:15.647351 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:15.647329    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xcmm7"
Oct 31 21:25:15.685503 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:15.685468    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-xcmm7"]
Oct 31 21:25:15.692425 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:15.692398    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Oct 31 21:25:15.748617 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:15.748580    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfw2x\" (UniqueName: \"kubernetes.io/projected/2c6009fc-0e3d-4178-84f3-eafc8245b15e-kube-api-access-kfw2x\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xcmm7\" (UID: \"2c6009fc-0e3d-4178-84f3-eafc8245b15e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xcmm7"
Oct 31 21:25:15.748798 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:15.748626    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c6009fc-0e3d-4178-84f3-eafc8245b15e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xcmm7\" (UID: \"2c6009fc-0e3d-4178-84f3-eafc8245b15e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xcmm7"
Oct 31 21:25:15.748798 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:15.748686    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c6009fc-0e3d-4178-84f3-eafc8245b15e-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xcmm7\" (UID: \"2c6009fc-0e3d-4178-84f3-eafc8245b15e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xcmm7"
Oct 31 21:25:15.748798 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:15.748753    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c6009fc-0e3d-4178-84f3-eafc8245b15e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xcmm7\" (UID: \"2c6009fc-0e3d-4178-84f3-eafc8245b15e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xcmm7"
Oct 31 21:25:15.748990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:15.748823    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c6009fc-0e3d-4178-84f3-eafc8245b15e-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xcmm7\" (UID: \"2c6009fc-0e3d-4178-84f3-eafc8245b15e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xcmm7"
Oct 31 21:25:15.748990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:15.748909    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c6009fc-0e3d-4178-84f3-eafc8245b15e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xcmm7\" (UID: \"2c6009fc-0e3d-4178-84f3-eafc8245b15e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xcmm7"
Oct 31 21:25:15.823759 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:15.823657    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:25:15.849713 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:15.849681    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfw2x\" (UniqueName: \"kubernetes.io/projected/2c6009fc-0e3d-4178-84f3-eafc8245b15e-kube-api-access-kfw2x\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xcmm7\" (UID: \"2c6009fc-0e3d-4178-84f3-eafc8245b15e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xcmm7"
Oct 31 21:25:15.849713 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:15.849715    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c6009fc-0e3d-4178-84f3-eafc8245b15e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xcmm7\" (UID: \"2c6009fc-0e3d-4178-84f3-eafc8245b15e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xcmm7"
Oct 31 21:25:15.849978 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:15.849746    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c6009fc-0e3d-4178-84f3-eafc8245b15e-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xcmm7\" (UID: \"2c6009fc-0e3d-4178-84f3-eafc8245b15e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xcmm7"
Oct 31 21:25:15.849978 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:15.849867    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c6009fc-0e3d-4178-84f3-eafc8245b15e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xcmm7\" (UID: \"2c6009fc-0e3d-4178-84f3-eafc8245b15e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xcmm7"
Oct 31 21:25:15.849978 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:15.849922    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c6009fc-0e3d-4178-84f3-eafc8245b15e-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xcmm7\" (UID: \"2c6009fc-0e3d-4178-84f3-eafc8245b15e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xcmm7"
Oct 31 21:25:15.849978 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:15.849974    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c6009fc-0e3d-4178-84f3-eafc8245b15e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xcmm7\" (UID: \"2c6009fc-0e3d-4178-84f3-eafc8245b15e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xcmm7"
Oct 31 21:25:15.850273 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:15.850244    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c6009fc-0e3d-4178-84f3-eafc8245b15e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xcmm7\" (UID: \"2c6009fc-0e3d-4178-84f3-eafc8245b15e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xcmm7"
Oct 31 21:25:15.850384 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:15.850288    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c6009fc-0e3d-4178-84f3-eafc8245b15e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xcmm7\" (UID: \"2c6009fc-0e3d-4178-84f3-eafc8245b15e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xcmm7"
Oct 31 21:25:15.850384 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:15.850310    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c6009fc-0e3d-4178-84f3-eafc8245b15e-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xcmm7\" (UID: \"2c6009fc-0e3d-4178-84f3-eafc8245b15e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xcmm7"
Oct 31 21:25:15.852001 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:15.851983    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c6009fc-0e3d-4178-84f3-eafc8245b15e-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xcmm7\" (UID: \"2c6009fc-0e3d-4178-84f3-eafc8245b15e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xcmm7"
Oct 31 21:25:15.852243 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:15.852228    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c6009fc-0e3d-4178-84f3-eafc8245b15e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xcmm7\" (UID: \"2c6009fc-0e3d-4178-84f3-eafc8245b15e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xcmm7"
Oct 31 21:25:15.875306 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:15.875276    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfw2x\" (UniqueName: \"kubernetes.io/projected/2c6009fc-0e3d-4178-84f3-eafc8245b15e-kube-api-access-kfw2x\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-xcmm7\" (UID: \"2c6009fc-0e3d-4178-84f3-eafc8245b15e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xcmm7"
Oct 31 21:25:15.960134 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:15.960094    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xcmm7"
Oct 31 21:25:16.123420 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:16.123352    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bnv8m"
Oct 31 21:25:16.263406 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:16.263378    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-xcmm7"]
Oct 31 21:25:16.265262 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:25:16.265235    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c6009fc_0e3d_4178_84f3_eafc8245b15e.slice/crio-6163d1cd97fc72cf8dd50e647ee8a16643220573415baef56c1cb950f25710fd WatchSource:0}: Error finding container 6163d1cd97fc72cf8dd50e647ee8a16643220573415baef56c1cb950f25710fd: Status 404 returned error can't find the container with id 6163d1cd97fc72cf8dd50e647ee8a16643220573415baef56c1cb950f25710fd
Oct 31 21:25:16.552367 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:16.552328    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xcmm7" event={"ID":"2c6009fc-0e3d-4178-84f3-eafc8245b15e","Type":"ContainerStarted","Data":"a5ad66bdf09372a1ea142998479c36d1bc0802cdf191db4c568abf0563a3a2fe"}
Oct 31 21:25:16.552838 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:16.552391    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xcmm7" event={"ID":"2c6009fc-0e3d-4178-84f3-eafc8245b15e","Type":"ContainerStarted","Data":"6163d1cd97fc72cf8dd50e647ee8a16643220573415baef56c1cb950f25710fd"}
Oct 31 21:25:16.598863 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:16.598812    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xcmm7" podStartSLOduration=1.430198982 podStartE2EDuration="1.598800189s" totalImagesPullingTime="168.601207ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:25:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:25:16.597441799 +0000 UTC m=+556.513751229" watchObservedRunningTime="2026-10-31 21:25:16.598800189 +0000 UTC m=+556.515109616"
Oct 31 21:25:16.960423 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:16.960383    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xcmm7"
Oct 31 21:25:18.495215 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:18.495175    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:25:19.831712 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:19.831678    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:25:21.449902 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:25:21.449867    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:25:21.691836 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:21.691803    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:25:22.738274 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:22.738239    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bfqc5"
Oct 31 21:25:22.750430 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:22.750406    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bfqc5"
Oct 31 21:25:23.121730 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:23.121699    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:25:23.597367 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:23.597334    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-bfqc5"
Oct 31 21:25:24.123047 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:24.122999    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bnv8m"
Oct 31 21:25:24.135040 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:24.135016    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bnv8m"
Oct 31 21:25:24.604661 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:24.604633    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-bnv8m"
Oct 31 21:25:25.821369 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:25.821336    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"]
Oct 31 21:25:25.824687 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:25.824668    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:25:25.857546 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:25.857522    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Oct 31 21:25:25.859078 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:25.859053    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"]
Oct 31 21:25:25.879670 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:25.879646    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:25:25.937782 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:25.937745    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzhcg\" (UniqueName: \"kubernetes.io/projected/9cd67bf5-8833-4a82-bc33-bed7bdfd4d36-kube-api-access-fzhcg\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p\" (UID: \"9cd67bf5-8833-4a82-bc33-bed7bdfd4d36\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:25:25.937918 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:25.937786    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cd67bf5-8833-4a82-bc33-bed7bdfd4d36-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p\" (UID: \"9cd67bf5-8833-4a82-bc33-bed7bdfd4d36\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:25:25.938002 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:25.937912    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cd67bf5-8833-4a82-bc33-bed7bdfd4d36-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p\" (UID: \"9cd67bf5-8833-4a82-bc33-bed7bdfd4d36\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:25:25.938041 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:25.938022    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cd67bf5-8833-4a82-bc33-bed7bdfd4d36-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p\" (UID: \"9cd67bf5-8833-4a82-bc33-bed7bdfd4d36\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:25:25.938081 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:25.938062    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cd67bf5-8833-4a82-bc33-bed7bdfd4d36-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p\" (UID: \"9cd67bf5-8833-4a82-bc33-bed7bdfd4d36\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:25:25.938126 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:25.938097    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cd67bf5-8833-4a82-bc33-bed7bdfd4d36-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p\" (UID: \"9cd67bf5-8833-4a82-bc33-bed7bdfd4d36\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:25:25.960686 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:25.960654    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xcmm7"
Oct 31 21:25:25.972858 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:25.972835    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xcmm7"
Oct 31 21:25:26.038831 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:26.038802    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzhcg\" (UniqueName: \"kubernetes.io/projected/9cd67bf5-8833-4a82-bc33-bed7bdfd4d36-kube-api-access-fzhcg\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p\" (UID: \"9cd67bf5-8833-4a82-bc33-bed7bdfd4d36\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:25:26.038831 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:26.038833    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cd67bf5-8833-4a82-bc33-bed7bdfd4d36-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p\" (UID: \"9cd67bf5-8833-4a82-bc33-bed7bdfd4d36\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:25:26.039084 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:26.038877    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cd67bf5-8833-4a82-bc33-bed7bdfd4d36-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p\" (UID: \"9cd67bf5-8833-4a82-bc33-bed7bdfd4d36\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:25:26.039084 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:26.038962    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cd67bf5-8833-4a82-bc33-bed7bdfd4d36-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p\" (UID: \"9cd67bf5-8833-4a82-bc33-bed7bdfd4d36\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:25:26.039084 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:26.039014    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cd67bf5-8833-4a82-bc33-bed7bdfd4d36-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p\" (UID: \"9cd67bf5-8833-4a82-bc33-bed7bdfd4d36\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:25:26.039084 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:26.039050    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cd67bf5-8833-4a82-bc33-bed7bdfd4d36-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p\" (UID: \"9cd67bf5-8833-4a82-bc33-bed7bdfd4d36\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:25:26.039358 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:26.039334    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cd67bf5-8833-4a82-bc33-bed7bdfd4d36-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p\" (UID: \"9cd67bf5-8833-4a82-bc33-bed7bdfd4d36\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:25:26.039435 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:26.039409    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cd67bf5-8833-4a82-bc33-bed7bdfd4d36-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p\" (UID: \"9cd67bf5-8833-4a82-bc33-bed7bdfd4d36\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:25:26.039483 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:26.039443    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cd67bf5-8833-4a82-bc33-bed7bdfd4d36-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p\" (UID: \"9cd67bf5-8833-4a82-bc33-bed7bdfd4d36\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:25:26.041373 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:26.041347    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cd67bf5-8833-4a82-bc33-bed7bdfd4d36-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p\" (UID: \"9cd67bf5-8833-4a82-bc33-bed7bdfd4d36\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:25:26.041463 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:26.041446    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cd67bf5-8833-4a82-bc33-bed7bdfd4d36-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p\" (UID: \"9cd67bf5-8833-4a82-bc33-bed7bdfd4d36\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:25:26.063619 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:26.063596    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzhcg\" (UniqueName: \"kubernetes.io/projected/9cd67bf5-8833-4a82-bc33-bed7bdfd4d36-kube-api-access-fzhcg\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p\" (UID: \"9cd67bf5-8833-4a82-bc33-bed7bdfd4d36\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:25:26.137366 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:26.137305    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:25:26.435148 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:26.435093    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"]
Oct 31 21:25:26.437416 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:25:26.437387    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9cd67bf5_8833_4a82_bc33_bed7bdfd4d36.slice/crio-51fe1dd9c2253df4f40bad35d881915c68c865f8fcb97c23bb98d95a6251a532 WatchSource:0}: Error finding container 51fe1dd9c2253df4f40bad35d881915c68c865f8fcb97c23bb98d95a6251a532: Status 404 returned error can't find the container with id 51fe1dd9c2253df4f40bad35d881915c68c865f8fcb97c23bb98d95a6251a532
Oct 31 21:25:26.555020 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:26.554988    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:25:26.600686 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:26.600650    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" event={"ID":"9cd67bf5-8833-4a82-bc33-bed7bdfd4d36","Type":"ContainerStarted","Data":"51fe1dd9c2253df4f40bad35d881915c68c865f8fcb97c23bb98d95a6251a532"}
Oct 31 21:25:26.611852 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:26.611830    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-xcmm7"
Oct 31 21:25:27.607322 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:27.607283    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" event={"ID":"9cd67bf5-8833-4a82-bc33-bed7bdfd4d36","Type":"ContainerStarted","Data":"d2d063e01596ee47dd3910eace0de19814fb334b886a8e0a8acc5a4c651086ca"}
Oct 31 21:25:28.137568 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:28.137536    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:25:34.059579 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:34.059521    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podStartSLOduration=8.874707893 podStartE2EDuration="9.059507151s" totalImagesPullingTime="184.799258ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:25:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:25:27.65286932 +0000 UTC m=+567.569178751" watchObservedRunningTime="2026-10-31 21:25:34.059507151 +0000 UTC m=+573.975816580"
Oct 31 21:25:34.061255 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:34.061236    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:25:36.137953 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:36.137913    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:25:36.150310 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:36.150279    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:25:36.657734 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:36.657703    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:25:43.972520 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:43.972445    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:25:45.034828 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:45.034787    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:25:46.493069 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:46.493027    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:25:46.625149 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:46.625117    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:25:49.960410 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:49.960371    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"]
Oct 31 21:25:49.963863 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:49.963841    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:25:49.974022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:49.973999    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"]
Oct 31 21:25:49.978626 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:49.978605    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"
Oct 31 21:25:49.992104 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:49.992079    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"]
Oct 31 21:25:49.997238 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:49.997216    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"]
Oct 31 21:25:50.048058 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.048028    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-94gsm\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:25:50.048058 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.048057    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-94gsm\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:25:50.048250 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.048076    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qt8c\" (UniqueName: \"kubernetes.io/projected/ae31c154-bb41-4ff0-aae3-93164dc757df-kube-api-access-2qt8c\") pod \"maas-default-gateway-openshift-default-58847f9fc5-94gsm\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:25:50.048250 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.048099    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0272849-0715-4e09-8d0b-827116fb35d5-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-hcd5s\" (UID: \"c0272849-0715-4e09-8d0b-827116fb35d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"
Oct 31 21:25:50.048250 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.048155    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae31c154-bb41-4ff0-aae3-93164dc757df-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-94gsm\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:25:50.048250 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.048240    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0272849-0715-4e09-8d0b-827116fb35d5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-hcd5s\" (UID: \"c0272849-0715-4e09-8d0b-827116fb35d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"
Oct 31 21:25:50.048421 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.048259    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0272849-0715-4e09-8d0b-827116fb35d5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-hcd5s\" (UID: \"c0272849-0715-4e09-8d0b-827116fb35d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"
Oct 31 21:25:50.048421 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.048339    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-94gsm\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:25:50.048421 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.048360    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0272849-0715-4e09-8d0b-827116fb35d5-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-hcd5s\" (UID: \"c0272849-0715-4e09-8d0b-827116fb35d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"
Oct 31 21:25:50.048421 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.048377    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g7cv\" (UniqueName: \"kubernetes.io/projected/c0272849-0715-4e09-8d0b-827116fb35d5-kube-api-access-7g7cv\") pod \"maas-default-gateway-openshift-default-58847f9fc5-hcd5s\" (UID: \"c0272849-0715-4e09-8d0b-827116fb35d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"
Oct 31 21:25:50.048421 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.048392    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-94gsm\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:25:50.048421 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.048417    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0272849-0715-4e09-8d0b-827116fb35d5-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-hcd5s\" (UID: \"c0272849-0715-4e09-8d0b-827116fb35d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"
Oct 31 21:25:50.048674 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.048475    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae31c154-bb41-4ff0-aae3-93164dc757df-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-94gsm\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:25:50.048674 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.048583    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0272849-0715-4e09-8d0b-827116fb35d5-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-hcd5s\" (UID: \"c0272849-0715-4e09-8d0b-827116fb35d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"
Oct 31 21:25:50.048674 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.048601    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-94gsm\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:25:50.048674 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.048623    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0272849-0715-4e09-8d0b-827116fb35d5-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-hcd5s\" (UID: \"c0272849-0715-4e09-8d0b-827116fb35d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"
Oct 31 21:25:50.048827 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.048679    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0272849-0715-4e09-8d0b-827116fb35d5-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-hcd5s\" (UID: \"c0272849-0715-4e09-8d0b-827116fb35d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"
Oct 31 21:25:50.048827 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.048703    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae31c154-bb41-4ff0-aae3-93164dc757df-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-94gsm\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:25:50.066930 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.066906    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-dn2rx\""
Oct 31 21:25:50.149309 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.149280    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0272849-0715-4e09-8d0b-827116fb35d5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-hcd5s\" (UID: \"c0272849-0715-4e09-8d0b-827116fb35d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"
Oct 31 21:25:50.149461 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.149316    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0272849-0715-4e09-8d0b-827116fb35d5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-hcd5s\" (UID: \"c0272849-0715-4e09-8d0b-827116fb35d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"
Oct 31 21:25:50.149461 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.149351    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-94gsm\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:25:50.149461 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.149377    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0272849-0715-4e09-8d0b-827116fb35d5-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-hcd5s\" (UID: \"c0272849-0715-4e09-8d0b-827116fb35d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"
Oct 31 21:25:50.149461 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.149403    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g7cv\" (UniqueName: \"kubernetes.io/projected/c0272849-0715-4e09-8d0b-827116fb35d5-kube-api-access-7g7cv\") pod \"maas-default-gateway-openshift-default-58847f9fc5-hcd5s\" (UID: \"c0272849-0715-4e09-8d0b-827116fb35d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"
Oct 31 21:25:50.149461 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.149428    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-94gsm\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:25:50.149461 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.149460    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0272849-0715-4e09-8d0b-827116fb35d5-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-hcd5s\" (UID: \"c0272849-0715-4e09-8d0b-827116fb35d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"
Oct 31 21:25:50.149755 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.149483    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae31c154-bb41-4ff0-aae3-93164dc757df-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-94gsm\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:25:50.149755 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.149560    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0272849-0715-4e09-8d0b-827116fb35d5-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-hcd5s\" (UID: \"c0272849-0715-4e09-8d0b-827116fb35d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"
Oct 31 21:25:50.149755 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.149586    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-94gsm\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:25:50.149755 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.149626    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0272849-0715-4e09-8d0b-827116fb35d5-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-hcd5s\" (UID: \"c0272849-0715-4e09-8d0b-827116fb35d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"
Oct 31 21:25:50.149755 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.149677    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0272849-0715-4e09-8d0b-827116fb35d5-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-hcd5s\" (UID: \"c0272849-0715-4e09-8d0b-827116fb35d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"
Oct 31 21:25:50.149755 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.149709    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae31c154-bb41-4ff0-aae3-93164dc757df-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-94gsm\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:25:50.150068 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.149765    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-94gsm\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:25:50.150068 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.149766    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-94gsm\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:25:50.150068 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.149792    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-94gsm\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:25:50.150068 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.149818    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qt8c\" (UniqueName: \"kubernetes.io/projected/ae31c154-bb41-4ff0-aae3-93164dc757df-kube-api-access-2qt8c\") pod \"maas-default-gateway-openshift-default-58847f9fc5-94gsm\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:25:50.150068 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.149852    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0272849-0715-4e09-8d0b-827116fb35d5-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-hcd5s\" (UID: \"c0272849-0715-4e09-8d0b-827116fb35d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"
Oct 31 21:25:50.150068 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.149864    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0272849-0715-4e09-8d0b-827116fb35d5-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-hcd5s\" (UID: \"c0272849-0715-4e09-8d0b-827116fb35d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"
Oct 31 21:25:50.150068 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.149893    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae31c154-bb41-4ff0-aae3-93164dc757df-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-94gsm\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:25:50.150068 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.149929    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-94gsm\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:25:50.150068 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.150062    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0272849-0715-4e09-8d0b-827116fb35d5-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-hcd5s\" (UID: \"c0272849-0715-4e09-8d0b-827116fb35d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"
Oct 31 21:25:50.150510 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.150294    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-94gsm\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:25:50.150510 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.150404    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0272849-0715-4e09-8d0b-827116fb35d5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-hcd5s\" (UID: \"c0272849-0715-4e09-8d0b-827116fb35d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"
Oct 31 21:25:50.150619 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.150570    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-94gsm\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:25:50.150619 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.150576    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae31c154-bb41-4ff0-aae3-93164dc757df-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-94gsm\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:25:50.150847 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.150824    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0272849-0715-4e09-8d0b-827116fb35d5-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-hcd5s\" (UID: \"c0272849-0715-4e09-8d0b-827116fb35d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"
Oct 31 21:25:50.150950 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.150878    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0272849-0715-4e09-8d0b-827116fb35d5-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-hcd5s\" (UID: \"c0272849-0715-4e09-8d0b-827116fb35d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"
Oct 31 21:25:50.152216 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.152194    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-94gsm\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:25:50.152335 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.152314    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0272849-0715-4e09-8d0b-827116fb35d5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-hcd5s\" (UID: \"c0272849-0715-4e09-8d0b-827116fb35d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"
Oct 31 21:25:50.152470 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.152453    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae31c154-bb41-4ff0-aae3-93164dc757df-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-94gsm\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:25:50.152534 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.152517    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0272849-0715-4e09-8d0b-827116fb35d5-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-hcd5s\" (UID: \"c0272849-0715-4e09-8d0b-827116fb35d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"
Oct 31 21:25:50.174554 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.174527    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0272849-0715-4e09-8d0b-827116fb35d5-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-hcd5s\" (UID: \"c0272849-0715-4e09-8d0b-827116fb35d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"
Oct 31 21:25:50.174646 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.174580    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g7cv\" (UniqueName: \"kubernetes.io/projected/c0272849-0715-4e09-8d0b-827116fb35d5-kube-api-access-7g7cv\") pod \"maas-default-gateway-openshift-default-58847f9fc5-hcd5s\" (UID: \"c0272849-0715-4e09-8d0b-827116fb35d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"
Oct 31 21:25:50.174805 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.174784    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qt8c\" (UniqueName: \"kubernetes.io/projected/ae31c154-bb41-4ff0-aae3-93164dc757df-kube-api-access-2qt8c\") pod \"maas-default-gateway-openshift-default-58847f9fc5-94gsm\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:25:50.174837 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.174802    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae31c154-bb41-4ff0-aae3-93164dc757df-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-94gsm\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:25:50.278906 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.278884    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:25:50.296169 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.296144    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"
Oct 31 21:25:50.613412 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.613388    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"]
Oct 31 21:25:50.615353 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:25:50.615326    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae31c154_bb41_4ff0_aae3_93164dc757df.slice/crio-6156aa29791be5302e7b26da7949c2c2bbe780683659aab95db49f17d4722300 WatchSource:0}: Error finding container 6156aa29791be5302e7b26da7949c2c2bbe780683659aab95db49f17d4722300: Status 404 returned error can't find the container with id 6156aa29791be5302e7b26da7949c2c2bbe780683659aab95db49f17d4722300
Oct 31 21:25:50.625969 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.625924    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"]
Oct 31 21:25:50.627852 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:25:50.627830    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0272849_0715_4e09_8d0b_827116fb35d5.slice/crio-61d5cc75b7c813a4e7102f025058246443ca1559eb050b481f62e183400c704c WatchSource:0}: Error finding container 61d5cc75b7c813a4e7102f025058246443ca1559eb050b481f62e183400c704c: Status 404 returned error can't find the container with id 61d5cc75b7c813a4e7102f025058246443ca1559eb050b481f62e183400c704c
Oct 31 21:25:50.716984 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.716921    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm" event={"ID":"ae31c154-bb41-4ff0-aae3-93164dc757df","Type":"ContainerStarted","Data":"6156aa29791be5302e7b26da7949c2c2bbe780683659aab95db49f17d4722300"}
Oct 31 21:25:50.718138 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:50.718111    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s" event={"ID":"c0272849-0715-4e09-8d0b-827116fb35d5","Type":"ContainerStarted","Data":"61d5cc75b7c813a4e7102f025058246443ca1559eb050b481f62e183400c704c"}
Oct 31 21:25:51.459460 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:25:51.459426    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:25:53.286367 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:53.286336    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:25:53.286655 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:53.286398    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:25:53.286655 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:53.286427    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:25:53.394797 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:53.394748    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:25:53.394919 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:53.394847    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:25:53.394919 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:53.394889    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:25:53.735977 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:53.735839    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm" event={"ID":"ae31c154-bb41-4ff0-aae3-93164dc757df","Type":"ContainerStarted","Data":"672fe7097d87651a9d95a6b43c036967cbd6bf929bcc17f04a0ca1dc35a4f4b9"}
Oct 31 21:25:53.737530 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:53.737505    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s" event={"ID":"c0272849-0715-4e09-8d0b-827116fb35d5","Type":"ContainerStarted","Data":"f05e2da8c79dd99d038713aa5132d0a14777d0ac8bd8216f15e3ed7a10050168"}
Oct 31 21:25:53.811634 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:53.811583    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm" podStartSLOduration=2.143185736 podStartE2EDuration="4.811569104s" totalImagesPullingTime="2.668383368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:25:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:25:53.797855209 +0000 UTC m=+593.714164640" watchObservedRunningTime="2026-10-31 21:25:53.811569104 +0000 UTC m=+593.727878608"
Oct 31 21:25:53.850919 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:53.850864    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s" podStartSLOduration=2.086701935 podStartE2EDuration="4.850849302s" totalImagesPullingTime="2.764147367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:25:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:25:53.849097467 +0000 UTC m=+593.765406886" watchObservedRunningTime="2026-10-31 21:25:53.850849302 +0000 UTC m=+593.767158730"
Oct 31 21:25:54.279537 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:54.279494    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:25:54.280990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:54.280964    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-94gsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Oct 31 21:25:54.281104 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:54.281008    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm" podUID="ae31c154-bb41-4ff0-aae3-93164dc757df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Oct 31 21:25:54.296486 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:54.296462    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"
Oct 31 21:25:54.297608 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:54.297585    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-hcd5s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 31 21:25:54.297683 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:54.297619    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s" podUID="c0272849-0715-4e09-8d0b-827116fb35d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 31 21:25:55.285074 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:55.285022    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-94gsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Oct 31 21:25:55.285392 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:55.285097    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm" podUID="ae31c154-bb41-4ff0-aae3-93164dc757df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Oct 31 21:25:55.297484 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:55.297454    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-hcd5s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 31 21:25:55.297825 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:55.297501    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s" podUID="c0272849-0715-4e09-8d0b-827116fb35d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 31 21:25:56.279920 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:56.279876    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-94gsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Oct 31 21:25:56.280130 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:56.279962    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm" podUID="ae31c154-bb41-4ff0-aae3-93164dc757df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Oct 31 21:25:56.297403 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:56.297362    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-hcd5s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 31 21:25:56.297484 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:56.297427    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s" podUID="c0272849-0715-4e09-8d0b-827116fb35d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 31 21:25:57.280011 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:57.279974    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-94gsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Oct 31 21:25:57.280439 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:57.280031    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm" podUID="ae31c154-bb41-4ff0-aae3-93164dc757df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Oct 31 21:25:57.296444 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:57.296414    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-hcd5s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 31 21:25:57.296555 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:57.296459    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s" podUID="c0272849-0715-4e09-8d0b-827116fb35d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 31 21:25:58.279410 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:58.279373    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-94gsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Oct 31 21:25:58.279589 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:58.279421    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm" podUID="ae31c154-bb41-4ff0-aae3-93164dc757df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Oct 31 21:25:58.297076 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:58.297042    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-hcd5s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 31 21:25:58.297405 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:58.297096    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s" podUID="c0272849-0715-4e09-8d0b-827116fb35d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 31 21:25:59.279263 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:59.279228    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-94gsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Oct 31 21:25:59.279447 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:59.279291    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm" podUID="ae31c154-bb41-4ff0-aae3-93164dc757df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Oct 31 21:25:59.297229 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:59.297201    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-hcd5s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 31 21:25:59.297514 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:25:59.297248    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s" podUID="c0272849-0715-4e09-8d0b-827116fb35d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 31 21:26:00.271475 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:00.271440    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ssl9_26069598-5dc4-4c96-b55f-dd2f09884419/ovn-acl-logging/0.log"
Oct 31 21:26:00.272978 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:00.272933    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ssl9_26069598-5dc4-4c96-b55f-dd2f09884419/ovn-acl-logging/0.log"
Oct 31 21:26:00.279236 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:00.279208    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-94gsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Oct 31 21:26:00.279417 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:00.279388    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm" podUID="ae31c154-bb41-4ff0-aae3-93164dc757df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Oct 31 21:26:00.296494 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:00.296468    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-hcd5s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Oct 31 21:26:00.296595 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:00.296517    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s" podUID="c0272849-0715-4e09-8d0b-827116fb35d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Oct 31 21:26:01.601509 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:01.601273    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:26:01.602171 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:01.602149    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:26:01.603069 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:01.603026    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:26:01.609915 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:01.609893    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"
Oct 31 21:26:01.610666 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:01.610648    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"
Oct 31 21:26:01.611258 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:01.611240    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s"
Oct 31 21:26:05.033955 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.033907    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"]
Oct 31 21:26:05.038665 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.038644    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:26:05.072291 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.072261    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"]
Oct 31 21:26:05.140026 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.139992    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-mnz78\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:26:05.140167 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.140043    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-mnz78\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:26:05.140167 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.140074    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e7288b0-58f2-41e8-a26e-eddac51598e0-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-mnz78\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:26:05.140269 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.140170    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-mnz78\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:26:05.140269 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.140240    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-mnz78\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:26:05.140351 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.140301    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xvhm\" (UniqueName: \"kubernetes.io/projected/7e7288b0-58f2-41e8-a26e-eddac51598e0-kube-api-access-7xvhm\") pod \"maas-default-gateway-openshift-default-58847f9fc5-mnz78\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:26:05.140351 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.140322    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-mnz78\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:26:05.140351 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.140340    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e7288b0-58f2-41e8-a26e-eddac51598e0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-mnz78\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:26:05.140468 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.140365    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e7288b0-58f2-41e8-a26e-eddac51598e0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-mnz78\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:26:05.241746 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.241714    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-mnz78\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:26:05.241910 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.241753    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e7288b0-58f2-41e8-a26e-eddac51598e0-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-mnz78\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:26:05.241910 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.241785    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-mnz78\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:26:05.241910 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.241811    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-mnz78\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:26:05.241910 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.241836    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xvhm\" (UniqueName: \"kubernetes.io/projected/7e7288b0-58f2-41e8-a26e-eddac51598e0-kube-api-access-7xvhm\") pod \"maas-default-gateway-openshift-default-58847f9fc5-mnz78\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:26:05.241910 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.241856    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-mnz78\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:26:05.241910 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.241873    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e7288b0-58f2-41e8-a26e-eddac51598e0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-mnz78\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:26:05.241910 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.241897    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e7288b0-58f2-41e8-a26e-eddac51598e0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-mnz78\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:26:05.242305 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.241990    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-mnz78\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:26:05.242305 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.242182    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-mnz78\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:26:05.242305 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.242194    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-mnz78\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:26:05.242742 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.242714    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e7288b0-58f2-41e8-a26e-eddac51598e0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-mnz78\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:26:05.242742 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.242733    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-mnz78\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:26:05.242871 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.242718    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-mnz78\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:26:05.244335 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.244310    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-mnz78\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:26:05.244435 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.244410    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e7288b0-58f2-41e8-a26e-eddac51598e0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-mnz78\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:26:05.267022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.266999    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xvhm\" (UniqueName: \"kubernetes.io/projected/7e7288b0-58f2-41e8-a26e-eddac51598e0-kube-api-access-7xvhm\") pod \"maas-default-gateway-openshift-default-58847f9fc5-mnz78\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:26:05.267114 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.267025    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e7288b0-58f2-41e8-a26e-eddac51598e0-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-mnz78\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:26:05.352971 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.352892    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:26:05.666917 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.666827    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:05.667367 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.667347    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"]
Oct 31 21:26:05.670239 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:26:05.670218    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e7288b0_58f2_41e8_a26e_eddac51598e0.slice/crio-48dfe40325087e6d6d2ca92599677fdee0559904bedfd56f8e5a6acc0544b1f0 WatchSource:0}: Error finding container 48dfe40325087e6d6d2ca92599677fdee0559904bedfd56f8e5a6acc0544b1f0: Status 404 returned error can't find the container with id 48dfe40325087e6d6d2ca92599677fdee0559904bedfd56f8e5a6acc0544b1f0
Oct 31 21:26:05.672123 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.672096    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:26:05.672186 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.672157    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:26:05.672221 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.672190    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:26:05.833415 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.833376    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78" event={"ID":"7e7288b0-58f2-41e8-a26e-eddac51598e0","Type":"ContainerStarted","Data":"849a42cd2a5e199c7923c92d4ee4a075cf62e57926d39bee73a112da58cab459"}
Oct 31 21:26:05.833692 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.833439    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78" event={"ID":"7e7288b0-58f2-41e8-a26e-eddac51598e0","Type":"ContainerStarted","Data":"48dfe40325087e6d6d2ca92599677fdee0559904bedfd56f8e5a6acc0544b1f0"}
Oct 31 21:26:05.883298 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:05.883241    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78" podStartSLOduration=0.883225937 podStartE2EDuration="883.225937ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:26:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:26:05.88097196 +0000 UTC m=+605.797281389" watchObservedRunningTime="2026-10-31 21:26:05.883225937 +0000 UTC m=+605.799535365"
Oct 31 21:26:06.353407 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:06.353369    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:26:06.354870 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:06.354844    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-mnz78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 31 21:26:06.354999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:06.354893    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78" podUID="7e7288b0-58f2-41e8-a26e-eddac51598e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 31 21:26:07.353517 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:07.353483    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-mnz78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 31 21:26:07.353925 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:07.353572    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78" podUID="7e7288b0-58f2-41e8-a26e-eddac51598e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 31 21:26:08.354085 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:08.354049    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-mnz78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 31 21:26:08.354466 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:08.354109    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78" podUID="7e7288b0-58f2-41e8-a26e-eddac51598e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 31 21:26:09.353448 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:09.353412    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-mnz78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 31 21:26:09.353616 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:09.353472    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78" podUID="7e7288b0-58f2-41e8-a26e-eddac51598e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 31 21:26:10.353306 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:10.353269    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-mnz78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 31 21:26:10.353695 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:10.353318    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78" podUID="7e7288b0-58f2-41e8-a26e-eddac51598e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 31 21:26:11.354132 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:11.354092    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-mnz78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 31 21:26:11.354520 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:11.354153    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78" podUID="7e7288b0-58f2-41e8-a26e-eddac51598e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 31 21:26:12.124738 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:12.124699    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:12.250338 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:12.250307    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:12.353527 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:12.353489    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-mnz78 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body=
Oct 31 21:26:12.353671 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:12.353548    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78" podUID="7e7288b0-58f2-41e8-a26e-eddac51598e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused"
Oct 31 21:26:13.669636 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:13.669596    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:26:13.670226 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:13.670204    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:26:13.670795 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:13.670772    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:26:14.730922 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:14.730883    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:14.843657 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:14.843620    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:14.961133 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:14.961095    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:15.082889 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.082840    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:15.270811 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.270764    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:15.319269 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.319236    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"]
Oct 31 21:26:15.323962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.323924    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:15.336760 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.336736    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istio-podinfo\") pod \"e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") " pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:15.336853 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.336785    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istio-token\") pod \"e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") " pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:15.336907 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.336888    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istiod-ca-cert\") pod \"e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") " pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:15.336959 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.336920    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-workload-socket\") pod \"e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") " pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:15.337006 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.336995    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istio-envoy\") pod \"e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") " pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:15.337057 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.337045    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-credential-socket\") pod \"e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") " pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:15.337114 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.337101    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kq4c7\" (UniqueName: \"kubernetes.io/projected/75599b5e-0a6e-47e0-8598-39e58ee2edfd-kube-api-access-kq4c7\") pod \"e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") " pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:15.337150 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.337130    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-workload-certs\") pod \"e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") " pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:15.337183 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.337151    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istio-data\") pod \"e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") " pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:15.341011 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.340992    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-ac0736b6-openshift-default-dockercfg-bxwgm\""
Oct 31 21:26:15.356592 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.356570    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"]
Oct 31 21:26:15.389332 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.389304    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:15.438474 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.438447    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istio-token\") pod \"e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") " pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:15.438647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.438486    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istiod-ca-cert\") pod \"e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") " pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:15.438647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.438504    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-workload-socket\") pod \"e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") " pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:15.438647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.438541    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istio-envoy\") pod \"e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") " pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:15.438647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.438583    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-credential-socket\") pod \"e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") " pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:15.438647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.438610    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kq4c7\" (UniqueName: \"kubernetes.io/projected/75599b5e-0a6e-47e0-8598-39e58ee2edfd-kube-api-access-kq4c7\") pod \"e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") " pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:15.438647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.438627    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-workload-certs\") pod \"e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") " pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:15.438647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.438648    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istio-data\") pod \"e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") " pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:15.439025 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.438719    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istio-podinfo\") pod \"e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") " pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:15.439025 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.439014    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-workload-socket\") pod \"e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") " pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:15.439135 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.439048    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-credential-socket\") pod \"e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") " pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:15.439135 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.439106    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istio-data\") pod \"e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") " pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:15.439234 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.439150    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-workload-certs\") pod \"e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") " pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:15.439291 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.439250    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istiod-ca-cert\") pod \"e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") " pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:15.440966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.440922    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istio-envoy\") pod \"e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") " pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:15.441306 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.441287    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istio-podinfo\") pod \"e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") " pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:15.471185 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.471155    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kq4c7\" (UniqueName: \"kubernetes.io/projected/75599b5e-0a6e-47e0-8598-39e58ee2edfd-kube-api-access-kq4c7\") pod \"e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") " pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:15.471312 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.471235    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istio-token\") pod \"e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") " pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:15.505855 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.505816    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:15.614918 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.614797    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:15.640706 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.640675    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:15.727167 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.724846    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:15.887141 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.887044    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:15.964585 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.964562    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"]
Oct 31 21:26:15.967154 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:26:15.967122    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75599b5e_0a6e_47e0_8598_39e58ee2edfd.slice/crio-495dfbe25d98ec544cfbcdc2f97d869c8f335348e1f252217f16c4a7e10fbabc WatchSource:0}: Error finding container 495dfbe25d98ec544cfbcdc2f97d869c8f335348e1f252217f16c4a7e10fbabc: Status 404 returned error can't find the container with id 495dfbe25d98ec544cfbcdc2f97d869c8f335348e1f252217f16c4a7e10fbabc
Oct 31 21:26:15.969589 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.969552    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:26:15.969669 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.969636    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:26:15.969705 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:15.969682    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:26:16.001620 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:16.001592    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:16.680374 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:16.680337    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:16.895786 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:16.895745    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2" event={"ID":"75599b5e-0a6e-47e0-8598-39e58ee2edfd","Type":"ContainerStarted","Data":"8ae36d9c22de57b2c2af20f3e4b4dbe3117d6280358b8470a8fc0a8ce7a30502"}
Oct 31 21:26:16.896192 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:16.895837    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2" event={"ID":"75599b5e-0a6e-47e0-8598-39e58ee2edfd","Type":"ContainerStarted","Data":"495dfbe25d98ec544cfbcdc2f97d869c8f335348e1f252217f16c4a7e10fbabc"}
Oct 31 21:26:16.945381 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:16.945293    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2" podStartSLOduration=1.945279719 podStartE2EDuration="1.945279719s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:26:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:26:16.943742433 +0000 UTC m=+616.860051863" watchObservedRunningTime="2026-10-31 21:26:16.945279719 +0000 UTC m=+616.861589147"
Oct 31 21:26:17.110289 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.110258    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"]
Oct 31 21:26:17.113913 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.113897    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:17.138888 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.138858    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"]
Oct 31 21:26:17.153493 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.153463    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-istio-data\") pod \"e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") " pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:17.153619 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.153506    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37956991-b640-4c37-acc6-888d3f4b1ef4-istio-token\") pod \"e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") " pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:17.153679 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.153657    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-workload-socket\") pod \"e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") " pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:17.153717 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.153705    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-credential-socket\") pod \"e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") " pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:17.153767 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.153756    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-workload-certs\") pod \"e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") " pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:17.153818 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.153782    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37956991-b640-4c37-acc6-888d3f4b1ef4-istiod-ca-cert\") pod \"e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") " pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:17.153871 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.153853    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjkgm\" (UniqueName: \"kubernetes.io/projected/37956991-b640-4c37-acc6-888d3f4b1ef4-kube-api-access-vjkgm\") pod \"e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") " pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:17.153968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.153950    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37956991-b640-4c37-acc6-888d3f4b1ef4-istio-podinfo\") pod \"e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") " pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:17.154013 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.153982    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-istio-envoy\") pod \"e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") " pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:17.171324 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.171290    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:17.233409 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.233320    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-9df039e4-openshift-default-dockercfg-fbfww\""
Oct 31 21:26:17.255822 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.254682    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-istio-data\") pod \"e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") " pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:17.255822 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.254737    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37956991-b640-4c37-acc6-888d3f4b1ef4-istio-token\") pod \"e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") " pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:17.255822 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.254787    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-workload-socket\") pod \"e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") " pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:17.255822 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.254845    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-credential-socket\") pod \"e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") " pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:17.255822 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.254896    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-workload-certs\") pod \"e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") " pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:17.255822 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.254954    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37956991-b640-4c37-acc6-888d3f4b1ef4-istiod-ca-cert\") pod \"e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") " pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:17.255822 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.254986    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjkgm\" (UniqueName: \"kubernetes.io/projected/37956991-b640-4c37-acc6-888d3f4b1ef4-kube-api-access-vjkgm\") pod \"e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") " pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:17.255822 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.255072    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37956991-b640-4c37-acc6-888d3f4b1ef4-istio-podinfo\") pod \"e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") " pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:17.255822 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.255105    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-istio-envoy\") pod \"e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") " pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:17.257680 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.257646    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37956991-b640-4c37-acc6-888d3f4b1ef4-istiod-ca-cert\") pod \"e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") " pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:17.259140 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.257907    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-workload-certs\") pod \"e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") " pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:17.259140 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.258182    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-istio-data\") pod \"e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") " pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:17.259573 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.259548    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-credential-socket\") pod \"e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") " pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:17.259659 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.259605    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-workload-socket\") pod \"e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") " pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:17.260352 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.260329    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-istio-envoy\") pod \"e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") " pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:17.261984 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.261921    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37956991-b640-4c37-acc6-888d3f4b1ef4-istio-podinfo\") pod \"e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") " pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:17.284917 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.284884    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjkgm\" (UniqueName: \"kubernetes.io/projected/37956991-b640-4c37-acc6-888d3f4b1ef4-kube-api-access-vjkgm\") pod \"e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") " pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:17.292607 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.292581    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37956991-b640-4c37-acc6-888d3f4b1ef4-istio-token\") pod \"e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") " pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:17.427108 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.427065    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:17.495309 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.495276    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:17.641258 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.641227    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:17.646142 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.646116    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:17.756752 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.756730    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"]
Oct 31 21:26:17.758470 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:26:17.758440    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37956991_b640_4c37_acc6_888d3f4b1ef4.slice/crio-df35047c31baf4a60a2006d4496e67868a7551e878c06d4dff835d9514636a23 WatchSource:0}: Error finding container df35047c31baf4a60a2006d4496e67868a7551e878c06d4dff835d9514636a23: Status 404 returned error can't find the container with id df35047c31baf4a60a2006d4496e67868a7551e878c06d4dff835d9514636a23
Oct 31 21:26:17.760788 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.760759    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:26:17.760867 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.760821    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:26:17.760867 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.760849    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:26:17.903380 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.903338    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq" event={"ID":"37956991-b640-4c37-acc6-888d3f4b1ef4","Type":"ContainerStarted","Data":"d9d1d5f30e9d671f43c60392e6ed5bcccc950cbd4c514577f184dad625ef6fe0"}
Oct 31 21:26:17.903857 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.903407    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq" event={"ID":"37956991-b640-4c37-acc6-888d3f4b1ef4","Type":"ContainerStarted","Data":"df35047c31baf4a60a2006d4496e67868a7551e878c06d4dff835d9514636a23"}
Oct 31 21:26:17.904009 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.903953    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:17.904625 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.904609    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:17.953277 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:17.953231    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq" podStartSLOduration=0.953217728 podStartE2EDuration="953.217728ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:26:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:26:17.951217892 +0000 UTC m=+617.867527322" watchObservedRunningTime="2026-10-31 21:26:17.953217728 +0000 UTC m=+617.869527157"
Oct 31 21:26:18.427202 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:18.427169    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:18.431898 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:18.431867    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:18.742932 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:18.742840    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:18.854408 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:18.854376    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:18.908584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:18.908552    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:18.909497 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:18.909480    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:18.977691 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:18.977656    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:19.088899 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:19.088860    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:19.219372 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:19.219339    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:19.333117 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:19.333078    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:19.981655 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:19.981610    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:20.127044 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:20.127005    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:20.267580 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:20.267552    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-353db9f5-597d5d686c-xklsb"]
Oct 31 21:26:20.269245 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:20.269219    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:20.271079 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:20.271064    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-353db9f5-597d5d686c-xklsb"
Oct 31 21:26:20.284117 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:20.284094    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gz5vb\" (UniqueName: \"kubernetes.io/projected/8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa-kube-api-access-gz5vb\") pod \"payload-pre-processing-e2e-migrate-353db9f5-597d5d686c-xklsb\" (UID: \"8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-353db9f5-597d5d686c-xklsb"
Oct 31 21:26:20.284199 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:20.284134    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-353db9f5-597d5d686c-xklsb\" (UID: \"8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-353db9f5-597d5d686c-xklsb"
Oct 31 21:26:20.292996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:20.292974    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-353db9f5-597d5d686c-xklsb"]
Oct 31 21:26:20.304503 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:20.304475    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-353db9f5\""
Oct 31 21:26:20.304746 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:20.304729    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-353db9f5-dockercfg-lrpdn\""
Oct 31 21:26:20.360921 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:20.360885    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-353db9f5-69b57cffb7-6wt7s"]
Oct 31 21:26:20.365924 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:20.365897    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-353db9f5-69b57cffb7-6wt7s"
Oct 31 21:26:20.386999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:20.385153    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5ssv\" (UniqueName: \"kubernetes.io/projected/2b2e0a10-c671-44b8-b748-9929dd416fe8-kube-api-access-v5ssv\") pod \"payload-processing-e2e-migrate-353db9f5-69b57cffb7-6wt7s\" (UID: \"2b2e0a10-c671-44b8-b748-9929dd416fe8\") " pod="openshift-ingress/payload-processing-e2e-migrate-353db9f5-69b57cffb7-6wt7s"
Oct 31 21:26:20.386999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:20.385298    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b2e0a10-c671-44b8-b748-9929dd416fe8-plugins-config-volume\") pod \"payload-processing-e2e-migrate-353db9f5-69b57cffb7-6wt7s\" (UID: \"2b2e0a10-c671-44b8-b748-9929dd416fe8\") " pod="openshift-ingress/payload-processing-e2e-migrate-353db9f5-69b57cffb7-6wt7s"
Oct 31 21:26:20.386999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:20.385382    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gz5vb\" (UniqueName: \"kubernetes.io/projected/8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa-kube-api-access-gz5vb\") pod \"payload-pre-processing-e2e-migrate-353db9f5-597d5d686c-xklsb\" (UID: \"8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-353db9f5-597d5d686c-xklsb"
Oct 31 21:26:20.386999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:20.385741    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-353db9f5-597d5d686c-xklsb\" (UID: \"8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-353db9f5-597d5d686c-xklsb"
Oct 31 21:26:20.386999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:20.386319    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-353db9f5-69b57cffb7-6wt7s"]
Oct 31 21:26:20.386999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:20.386565    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-353db9f5-597d5d686c-xklsb\" (UID: \"8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-353db9f5-597d5d686c-xklsb"
Oct 31 21:26:20.410894 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:20.410870    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz5vb\" (UniqueName: \"kubernetes.io/projected/8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa-kube-api-access-gz5vb\") pod \"payload-pre-processing-e2e-migrate-353db9f5-597d5d686c-xklsb\" (UID: \"8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-353db9f5-597d5d686c-xklsb"
Oct 31 21:26:20.486885 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:20.486857    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b2e0a10-c671-44b8-b748-9929dd416fe8-plugins-config-volume\") pod \"payload-processing-e2e-migrate-353db9f5-69b57cffb7-6wt7s\" (UID: \"2b2e0a10-c671-44b8-b748-9929dd416fe8\") " pod="openshift-ingress/payload-processing-e2e-migrate-353db9f5-69b57cffb7-6wt7s"
Oct 31 21:26:20.487071 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:20.486928    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5ssv\" (UniqueName: \"kubernetes.io/projected/2b2e0a10-c671-44b8-b748-9929dd416fe8-kube-api-access-v5ssv\") pod \"payload-processing-e2e-migrate-353db9f5-69b57cffb7-6wt7s\" (UID: \"2b2e0a10-c671-44b8-b748-9929dd416fe8\") " pod="openshift-ingress/payload-processing-e2e-migrate-353db9f5-69b57cffb7-6wt7s"
Oct 31 21:26:20.487588 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:20.487568    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b2e0a10-c671-44b8-b748-9929dd416fe8-plugins-config-volume\") pod \"payload-processing-e2e-migrate-353db9f5-69b57cffb7-6wt7s\" (UID: \"2b2e0a10-c671-44b8-b748-9929dd416fe8\") " pod="openshift-ingress/payload-processing-e2e-migrate-353db9f5-69b57cffb7-6wt7s"
Oct 31 21:26:20.512033 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:20.512010    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5ssv\" (UniqueName: \"kubernetes.io/projected/2b2e0a10-c671-44b8-b748-9929dd416fe8-kube-api-access-v5ssv\") pod \"payload-processing-e2e-migrate-353db9f5-69b57cffb7-6wt7s\" (UID: \"2b2e0a10-c671-44b8-b748-9929dd416fe8\") " pod="openshift-ingress/payload-processing-e2e-migrate-353db9f5-69b57cffb7-6wt7s"
Oct 31 21:26:20.584163 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:20.584092    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-353db9f5-597d5d686c-xklsb"
Oct 31 21:26:20.685661 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:20.685627    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-353db9f5-69b57cffb7-6wt7s"
Oct 31 21:26:20.890815 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:20.890792    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-353db9f5-597d5d686c-xklsb"]
Oct 31 21:26:20.892364 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:26:20.892336    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f44fe4f_9c3c_4519_bcb1_3f21a1c6a8aa.slice/crio-3bffaaa582f710b7523c83f2702e6533f559a162bdebc1a8ad534a8a38041f84 WatchSource:0}: Error finding container 3bffaaa582f710b7523c83f2702e6533f559a162bdebc1a8ad534a8a38041f84: Status 404 returned error can't find the container with id 3bffaaa582f710b7523c83f2702e6533f559a162bdebc1a8ad534a8a38041f84
Oct 31 21:26:20.919999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:20.919966    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-353db9f5-597d5d686c-xklsb" event={"ID":"8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa","Type":"ContainerStarted","Data":"3bffaaa582f710b7523c83f2702e6533f559a162bdebc1a8ad534a8a38041f84"}
Oct 31 21:26:20.985638 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:20.985614    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-353db9f5-69b57cffb7-6wt7s"]
Oct 31 21:26:20.987989 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:26:20.987960    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b2e0a10_c671_44b8_b748_9929dd416fe8.slice/crio-1336a7a2a15d8d4bd5e5868bfc7c027cc66a172fab04a26016ed1f136d51ed0b WatchSource:0}: Error finding container 1336a7a2a15d8d4bd5e5868bfc7c027cc66a172fab04a26016ed1f136d51ed0b: Status 404 returned error can't find the container with id 1336a7a2a15d8d4bd5e5868bfc7c027cc66a172fab04a26016ed1f136d51ed0b
Oct 31 21:26:21.450585 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:26:21.450536    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:26:21.804622 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:21.804588    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:21.921191 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:21.921145    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:21.926369 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:21.926334    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-353db9f5-597d5d686c-xklsb" event={"ID":"8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa","Type":"ContainerStarted","Data":"b7522d6edf7b39c8f713b4f4536d1482b9e7356c1bf45583d038f811967f22a8"}
Oct 31 21:26:21.926544 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:21.926404    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-353db9f5-597d5d686c-xklsb"
Oct 31 21:26:21.927811 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:21.927786    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-353db9f5-69b57cffb7-6wt7s" event={"ID":"2b2e0a10-c671-44b8-b748-9929dd416fe8","Type":"ContainerStarted","Data":"0fe1f022ddceefd8b68b35b6f4cddd000459ba1aa9089dcd5f78b0f1d2e7bd23"}
Oct 31 21:26:21.927898 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:21.927831    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-353db9f5-69b57cffb7-6wt7s" event={"ID":"2b2e0a10-c671-44b8-b748-9929dd416fe8","Type":"ContainerStarted","Data":"1336a7a2a15d8d4bd5e5868bfc7c027cc66a172fab04a26016ed1f136d51ed0b"}
Oct 31 21:26:21.927898 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:21.927861    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-353db9f5-69b57cffb7-6wt7s"
Oct 31 21:26:21.963480 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:21.963451    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-25447e22-569c6647cf-ql62d"]
Oct 31 21:26:21.967099 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:21.967082    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-25447e22-569c6647cf-ql62d"
Oct 31 21:26:21.980713 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:21.980673    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-353db9f5-597d5d686c-xklsb" podStartSLOduration=1.980661679 podStartE2EDuration="1.980661679s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:26:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:26:21.978501923 +0000 UTC m=+621.894811352" watchObservedRunningTime="2026-10-31 21:26:21.980661679 +0000 UTC m=+621.896971108"
Oct 31 21:26:21.986481 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:21.986458    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-25447e22-dockercfg-2nz6d\""
Oct 31 21:26:21.992194 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:21.992172    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-25447e22\""
Oct 31 21:26:21.994556 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:21.994535    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-25447e22-569c6647cf-ql62d"]
Oct 31 21:26:21.999105 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:21.999080    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8875cc55-d9a9-4759-add2-2ce8d59a4a02-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-25447e22-569c6647cf-ql62d\" (UID: \"8875cc55-d9a9-4759-add2-2ce8d59a4a02\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-25447e22-569c6647cf-ql62d"
Oct 31 21:26:21.999286 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:21.999265    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wndcx\" (UniqueName: \"kubernetes.io/projected/8875cc55-d9a9-4759-add2-2ce8d59a4a02-kube-api-access-wndcx\") pod \"payload-pre-processing-e2e-ait-25447e22-569c6647cf-ql62d\" (UID: \"8875cc55-d9a9-4759-add2-2ce8d59a4a02\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-25447e22-569c6647cf-ql62d"
Oct 31 21:26:22.023019 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:22.022962    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-353db9f5-69b57cffb7-6wt7s" podStartSLOduration=2.022949107 podStartE2EDuration="2.022949107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:26:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:26:22.022308797 +0000 UTC m=+621.938618227" watchObservedRunningTime="2026-10-31 21:26:22.022949107 +0000 UTC m=+621.939258541"
Oct 31 21:26:22.034100 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:22.034069    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:22.064163 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:22.064096    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-25447e22-5998f5c689-sxqzp"]
Oct 31 21:26:22.068173 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:22.068153    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-25447e22-5998f5c689-sxqzp"
Oct 31 21:26:22.088179 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:22.088153    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-25447e22-5998f5c689-sxqzp"]
Oct 31 21:26:22.099798 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:22.099772    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wndcx\" (UniqueName: \"kubernetes.io/projected/8875cc55-d9a9-4759-add2-2ce8d59a4a02-kube-api-access-wndcx\") pod \"payload-pre-processing-e2e-ait-25447e22-569c6647cf-ql62d\" (UID: \"8875cc55-d9a9-4759-add2-2ce8d59a4a02\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-25447e22-569c6647cf-ql62d"
Oct 31 21:26:22.099966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:22.099806    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgn97\" (UniqueName: \"kubernetes.io/projected/fd4777f2-6998-443b-b61c-ce07542d48f4-kube-api-access-mgn97\") pod \"payload-processing-e2e-ait-25447e22-5998f5c689-sxqzp\" (UID: \"fd4777f2-6998-443b-b61c-ce07542d48f4\") " pod="openshift-ingress/payload-processing-e2e-ait-25447e22-5998f5c689-sxqzp"
Oct 31 21:26:22.099966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:22.099870    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8875cc55-d9a9-4759-add2-2ce8d59a4a02-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-25447e22-569c6647cf-ql62d\" (UID: \"8875cc55-d9a9-4759-add2-2ce8d59a4a02\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-25447e22-569c6647cf-ql62d"
Oct 31 21:26:22.100095 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:22.099965    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd4777f2-6998-443b-b61c-ce07542d48f4-plugins-config-volume\") pod \"payload-processing-e2e-ait-25447e22-5998f5c689-sxqzp\" (UID: \"fd4777f2-6998-443b-b61c-ce07542d48f4\") " pod="openshift-ingress/payload-processing-e2e-ait-25447e22-5998f5c689-sxqzp"
Oct 31 21:26:22.100593 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:22.100571    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8875cc55-d9a9-4759-add2-2ce8d59a4a02-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-25447e22-569c6647cf-ql62d\" (UID: \"8875cc55-d9a9-4759-add2-2ce8d59a4a02\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-25447e22-569c6647cf-ql62d"
Oct 31 21:26:22.124766 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:22.124742    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wndcx\" (UniqueName: \"kubernetes.io/projected/8875cc55-d9a9-4759-add2-2ce8d59a4a02-kube-api-access-wndcx\") pod \"payload-pre-processing-e2e-ait-25447e22-569c6647cf-ql62d\" (UID: \"8875cc55-d9a9-4759-add2-2ce8d59a4a02\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-25447e22-569c6647cf-ql62d"
Oct 31 21:26:22.201518 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:22.201465    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgn97\" (UniqueName: \"kubernetes.io/projected/fd4777f2-6998-443b-b61c-ce07542d48f4-kube-api-access-mgn97\") pod \"payload-processing-e2e-ait-25447e22-5998f5c689-sxqzp\" (UID: \"fd4777f2-6998-443b-b61c-ce07542d48f4\") " pod="openshift-ingress/payload-processing-e2e-ait-25447e22-5998f5c689-sxqzp"
Oct 31 21:26:22.201711 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:22.201590    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd4777f2-6998-443b-b61c-ce07542d48f4-plugins-config-volume\") pod \"payload-processing-e2e-ait-25447e22-5998f5c689-sxqzp\" (UID: \"fd4777f2-6998-443b-b61c-ce07542d48f4\") " pod="openshift-ingress/payload-processing-e2e-ait-25447e22-5998f5c689-sxqzp"
Oct 31 21:26:22.202367 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:22.202347    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd4777f2-6998-443b-b61c-ce07542d48f4-plugins-config-volume\") pod \"payload-processing-e2e-ait-25447e22-5998f5c689-sxqzp\" (UID: \"fd4777f2-6998-443b-b61c-ce07542d48f4\") " pod="openshift-ingress/payload-processing-e2e-ait-25447e22-5998f5c689-sxqzp"
Oct 31 21:26:22.226769 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:22.226743    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgn97\" (UniqueName: \"kubernetes.io/projected/fd4777f2-6998-443b-b61c-ce07542d48f4-kube-api-access-mgn97\") pod \"payload-processing-e2e-ait-25447e22-5998f5c689-sxqzp\" (UID: \"fd4777f2-6998-443b-b61c-ce07542d48f4\") " pod="openshift-ingress/payload-processing-e2e-ait-25447e22-5998f5c689-sxqzp"
Oct 31 21:26:22.280741 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:22.280702    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-25447e22-569c6647cf-ql62d"
Oct 31 21:26:22.381411 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:22.381382    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-25447e22-5998f5c689-sxqzp"
Oct 31 21:26:22.586441 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:22.586409    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-25447e22-569c6647cf-ql62d"]
Oct 31 21:26:22.588385 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:26:22.588359    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8875cc55_d9a9_4759_add2_2ce8d59a4a02.slice/crio-347298440c16e2fbbf4be1c10cddc0eec55f828fc53e8dde89f54d7915f70db0 WatchSource:0}: Error finding container 347298440c16e2fbbf4be1c10cddc0eec55f828fc53e8dde89f54d7915f70db0: Status 404 returned error can't find the container with id 347298440c16e2fbbf4be1c10cddc0eec55f828fc53e8dde89f54d7915f70db0
Oct 31 21:26:22.681072 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:22.681048    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-25447e22-5998f5c689-sxqzp"]
Oct 31 21:26:22.682714 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:26:22.682685    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd4777f2_6998_443b_b61c_ce07542d48f4.slice/crio-9acae83c548d653d62563279f28f4f39c0afe00cf92028ea65dfa5656746e95f WatchSource:0}: Error finding container 9acae83c548d653d62563279f28f4f39c0afe00cf92028ea65dfa5656746e95f: Status 404 returned error can't find the container with id 9acae83c548d653d62563279f28f4f39c0afe00cf92028ea65dfa5656746e95f
Oct 31 21:26:22.759107 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:22.759070    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:22.878154 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:22.878073    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:22.934436 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:22.934400    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-25447e22-569c6647cf-ql62d" event={"ID":"8875cc55-d9a9-4759-add2-2ce8d59a4a02","Type":"ContainerStarted","Data":"6709243c32a3c5f3a2bb5615300fd42cc5547482c7548aedb166a98465faabed"}
Oct 31 21:26:22.934620 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:22.934464    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-25447e22-569c6647cf-ql62d" event={"ID":"8875cc55-d9a9-4759-add2-2ce8d59a4a02","Type":"ContainerStarted","Data":"347298440c16e2fbbf4be1c10cddc0eec55f828fc53e8dde89f54d7915f70db0"}
Oct 31 21:26:22.934620 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:22.934494    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-25447e22-569c6647cf-ql62d"
Oct 31 21:26:22.936179 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:22.936150    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-25447e22-5998f5c689-sxqzp" event={"ID":"fd4777f2-6998-443b-b61c-ce07542d48f4","Type":"ContainerStarted","Data":"e3105f6450be35591d53a2bc68cce19897365ae1cd37e93687fdd50d80144060"}
Oct 31 21:26:22.936296 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:22.936212    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-25447e22-5998f5c689-sxqzp" event={"ID":"fd4777f2-6998-443b-b61c-ce07542d48f4","Type":"ContainerStarted","Data":"9acae83c548d653d62563279f28f4f39c0afe00cf92028ea65dfa5656746e95f"}
Oct 31 21:26:22.936296 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:22.936259    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-25447e22-5998f5c689-sxqzp"
Oct 31 21:26:22.980235 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:22.980191    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-25447e22-569c6647cf-ql62d" podStartSLOduration=1.980177838 podStartE2EDuration="1.980177838s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:26:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:26:22.979045534 +0000 UTC m=+622.895354977" watchObservedRunningTime="2026-10-31 21:26:22.980177838 +0000 UTC m=+622.896487301"
Oct 31 21:26:23.025356 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:23.025309    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-25447e22-5998f5c689-sxqzp" podStartSLOduration=1.025294408 podStartE2EDuration="1.025294408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:26:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:26:23.023827252 +0000 UTC m=+622.940136682" watchObservedRunningTime="2026-10-31 21:26:23.025294408 +0000 UTC m=+622.941603888"
Oct 31 21:26:23.222450 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:23.222352    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:23.335416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:23.335379    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:23.543873 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:23.543829    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:23.691854 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:23.691820    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9df039e4-5f997854db-wbbnx"]
Oct 31 21:26:23.695529 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:23.695509    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9df039e4-5f997854db-wbbnx"
Oct 31 21:26:23.725371 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:23.725333    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f94f57bd-1c2d-420b-ac9f-0806a79335e0-openshift-service-ca\") pod \"maas-api-e2e-ait-9df039e4-5f997854db-wbbnx\" (UID: \"f94f57bd-1c2d-420b-ac9f-0806a79335e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9df039e4-5f997854db-wbbnx"
Oct 31 21:26:23.725486 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:23.725415    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f94f57bd-1c2d-420b-ac9f-0806a79335e0-metrics-certs\") pod \"maas-api-e2e-ait-9df039e4-5f997854db-wbbnx\" (UID: \"f94f57bd-1c2d-420b-ac9f-0806a79335e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9df039e4-5f997854db-wbbnx"
Oct 31 21:26:23.725486 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:23.725479    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7ndq\" (UniqueName: \"kubernetes.io/projected/f94f57bd-1c2d-420b-ac9f-0806a79335e0-kube-api-access-q7ndq\") pod \"maas-api-e2e-ait-9df039e4-5f997854db-wbbnx\" (UID: \"f94f57bd-1c2d-420b-ac9f-0806a79335e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9df039e4-5f997854db-wbbnx"
Oct 31 21:26:23.725583 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:23.725545    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f94f57bd-1c2d-420b-ac9f-0806a79335e0-maas-api-tls\") pod \"maas-api-e2e-ait-9df039e4-5f997854db-wbbnx\" (UID: \"f94f57bd-1c2d-420b-ac9f-0806a79335e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9df039e4-5f997854db-wbbnx"
Oct 31 21:26:23.728232 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:23.728215    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-9df039e4-serving-cert\""
Oct 31 21:26:23.731524 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:23.731503    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9df039e4-5f997854db-wbbnx"]
Oct 31 21:26:23.792610 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:23.792580    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:23.826180 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:23.826107    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f94f57bd-1c2d-420b-ac9f-0806a79335e0-maas-api-tls\") pod \"maas-api-e2e-ait-9df039e4-5f997854db-wbbnx\" (UID: \"f94f57bd-1c2d-420b-ac9f-0806a79335e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9df039e4-5f997854db-wbbnx"
Oct 31 21:26:23.826316 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:23.826199    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f94f57bd-1c2d-420b-ac9f-0806a79335e0-openshift-service-ca\") pod \"maas-api-e2e-ait-9df039e4-5f997854db-wbbnx\" (UID: \"f94f57bd-1c2d-420b-ac9f-0806a79335e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9df039e4-5f997854db-wbbnx"
Oct 31 21:26:23.826316 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:23.826239    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f94f57bd-1c2d-420b-ac9f-0806a79335e0-metrics-certs\") pod \"maas-api-e2e-ait-9df039e4-5f997854db-wbbnx\" (UID: \"f94f57bd-1c2d-420b-ac9f-0806a79335e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9df039e4-5f997854db-wbbnx"
Oct 31 21:26:23.826316 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:26:23.826292    4460 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-9df039e4-serving-cert: secret "maas-api-e2e-ait-9df039e4-serving-cert" not found
Oct 31 21:26:23.826467 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:23.826360    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7ndq\" (UniqueName: \"kubernetes.io/projected/f94f57bd-1c2d-420b-ac9f-0806a79335e0-kube-api-access-q7ndq\") pod \"maas-api-e2e-ait-9df039e4-5f997854db-wbbnx\" (UID: \"f94f57bd-1c2d-420b-ac9f-0806a79335e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9df039e4-5f997854db-wbbnx"
Oct 31 21:26:23.826467 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:26:23.826387    4460 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f94f57bd-1c2d-420b-ac9f-0806a79335e0-maas-api-tls podName:f94f57bd-1c2d-420b-ac9f-0806a79335e0 nodeName:}" failed. No retries permitted until 2026-10-31 21:26:24.326365861 +0000 UTC m=+624.242675283 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f94f57bd-1c2d-420b-ac9f-0806a79335e0-maas-api-tls") pod "maas-api-e2e-ait-9df039e4-5f997854db-wbbnx" (UID: "f94f57bd-1c2d-420b-ac9f-0806a79335e0") : secret "maas-api-e2e-ait-9df039e4-serving-cert" not found
Oct 31 21:26:23.826917 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:23.826893    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f94f57bd-1c2d-420b-ac9f-0806a79335e0-openshift-service-ca\") pod \"maas-api-e2e-ait-9df039e4-5f997854db-wbbnx\" (UID: \"f94f57bd-1c2d-420b-ac9f-0806a79335e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9df039e4-5f997854db-wbbnx"
Oct 31 21:26:23.828700 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:23.828677    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f94f57bd-1c2d-420b-ac9f-0806a79335e0-metrics-certs\") pod \"maas-api-e2e-ait-9df039e4-5f997854db-wbbnx\" (UID: \"f94f57bd-1c2d-420b-ac9f-0806a79335e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9df039e4-5f997854db-wbbnx"
Oct 31 21:26:23.857300 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:23.857270    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7ndq\" (UniqueName: \"kubernetes.io/projected/f94f57bd-1c2d-420b-ac9f-0806a79335e0-kube-api-access-q7ndq\") pod \"maas-api-e2e-ait-9df039e4-5f997854db-wbbnx\" (UID: \"f94f57bd-1c2d-420b-ac9f-0806a79335e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9df039e4-5f997854db-wbbnx"
Oct 31 21:26:23.917633 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:23.917594    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:24.034880 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:24.034819    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:24.194960 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:24.194860    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:24.336844 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:24.336811    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f94f57bd-1c2d-420b-ac9f-0806a79335e0-maas-api-tls\") pod \"maas-api-e2e-ait-9df039e4-5f997854db-wbbnx\" (UID: \"f94f57bd-1c2d-420b-ac9f-0806a79335e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9df039e4-5f997854db-wbbnx"
Oct 31 21:26:24.339244 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:24.339211    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f94f57bd-1c2d-420b-ac9f-0806a79335e0-maas-api-tls\") pod \"maas-api-e2e-ait-9df039e4-5f997854db-wbbnx\" (UID: \"f94f57bd-1c2d-420b-ac9f-0806a79335e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-9df039e4-5f997854db-wbbnx"
Oct 31 21:26:24.608214 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:24.608183    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9df039e4-5f997854db-wbbnx"
Oct 31 21:26:24.913116 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:24.913093    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9df039e4-5f997854db-wbbnx"]
Oct 31 21:26:24.915160 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:26:24.915126    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf94f57bd_1c2d_420b_ac9f_0806a79335e0.slice/crio-93b992c90732a79580655b9a812e37ce966dcaf9ea5eb952f20c5f02056e0bd4 WatchSource:0}: Error finding container 93b992c90732a79580655b9a812e37ce966dcaf9ea5eb952f20c5f02056e0bd4: Status 404 returned error can't find the container with id 93b992c90732a79580655b9a812e37ce966dcaf9ea5eb952f20c5f02056e0bd4
Oct 31 21:26:24.916627 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:24.916608    4460 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 21:26:24.947003 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:24.946974    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9df039e4-5f997854db-wbbnx" event={"ID":"f94f57bd-1c2d-420b-ac9f-0806a79335e0","Type":"ContainerStarted","Data":"93b992c90732a79580655b9a812e37ce966dcaf9ea5eb952f20c5f02056e0bd4"}
Oct 31 21:26:25.155514 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:25.155466    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:25.272305 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:25.272271    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:25.327369 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:25.327338    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9df039e4-866c7df7b7-56zff"]
Oct 31 21:26:25.331161 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:25.331138    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-9df039e4-866c7df7b7-56zff"
Oct 31 21:26:25.354216 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:25.353702    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-9df039e4\""
Oct 31 21:26:25.354216 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:25.353925    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-9df039e4-dockercfg-jlftl\""
Oct 31 21:26:25.356183 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:25.356159    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9df039e4-866c7df7b7-56zff"]
Oct 31 21:26:25.455464 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:25.455419    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4d6l\" (UniqueName: \"kubernetes.io/projected/980f13c4-ba5f-4964-a3ef-2835f6916b99-kube-api-access-m4d6l\") pod \"payload-pre-processing-e2e-ait-9df039e4-866c7df7b7-56zff\" (UID: \"980f13c4-ba5f-4964-a3ef-2835f6916b99\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9df039e4-866c7df7b7-56zff"
Oct 31 21:26:25.455635 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:25.455563    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/980f13c4-ba5f-4964-a3ef-2835f6916b99-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-9df039e4-866c7df7b7-56zff\" (UID: \"980f13c4-ba5f-4964-a3ef-2835f6916b99\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9df039e4-866c7df7b7-56zff"
Oct 31 21:26:25.557176 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:25.557143    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/980f13c4-ba5f-4964-a3ef-2835f6916b99-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-9df039e4-866c7df7b7-56zff\" (UID: \"980f13c4-ba5f-4964-a3ef-2835f6916b99\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9df039e4-866c7df7b7-56zff"
Oct 31 21:26:25.557326 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:25.557229    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4d6l\" (UniqueName: \"kubernetes.io/projected/980f13c4-ba5f-4964-a3ef-2835f6916b99-kube-api-access-m4d6l\") pod \"payload-pre-processing-e2e-ait-9df039e4-866c7df7b7-56zff\" (UID: \"980f13c4-ba5f-4964-a3ef-2835f6916b99\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9df039e4-866c7df7b7-56zff"
Oct 31 21:26:25.557845 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:25.557823    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/980f13c4-ba5f-4964-a3ef-2835f6916b99-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-9df039e4-866c7df7b7-56zff\" (UID: \"980f13c4-ba5f-4964-a3ef-2835f6916b99\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9df039e4-866c7df7b7-56zff"
Oct 31 21:26:25.564325 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:25.564285    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:25.582138 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:25.582114    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4d6l\" (UniqueName: \"kubernetes.io/projected/980f13c4-ba5f-4964-a3ef-2835f6916b99-kube-api-access-m4d6l\") pod \"payload-pre-processing-e2e-ait-9df039e4-866c7df7b7-56zff\" (UID: \"980f13c4-ba5f-4964-a3ef-2835f6916b99\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-9df039e4-866c7df7b7-56zff"
Oct 31 21:26:25.643389 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:25.643354    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-9df039e4-866c7df7b7-56zff"
Oct 31 21:26:25.867604 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:25.865589    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:26:25.945629 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:25.945607    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9df039e4-866c7df7b7-56zff"]
Oct 31 21:26:25.948016 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:26:25.947978    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod980f13c4_ba5f_4964_a3ef_2835f6916b99.slice/crio-b1cdc81696a65692eafd1af017c28c72bfbb5299e763f25cdc1e8f34f2ab588f WatchSource:0}: Error finding container b1cdc81696a65692eafd1af017c28c72bfbb5299e763f25cdc1e8f34f2ab588f: Status 404 returned error can't find the container with id b1cdc81696a65692eafd1af017c28c72bfbb5299e763f25cdc1e8f34f2ab588f
Oct 31 21:26:25.954380 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:25.954355    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9df039e4-5f997854db-wbbnx" event={"ID":"f94f57bd-1c2d-420b-ac9f-0806a79335e0","Type":"ContainerStarted","Data":"23c4b717584a910177a9c81a3562321d06ed11f9f95fb350e3c599a7ab3af8a5"}
Oct 31 21:26:25.954526 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:25.954497    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9df039e4-5f997854db-wbbnx"
Oct 31 21:26:26.009485 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:26.009361    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9df039e4-5f997854db-wbbnx" podStartSLOduration=2.700129908 podStartE2EDuration="3.009340418s" totalImagesPullingTime="309.21051ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:26:26.007990999 +0000 UTC m=+625.924300430" watchObservedRunningTime="2026-10-31 21:26:26.009340418 +0000 UTC m=+625.925649848"
Oct 31 21:26:26.855098 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:26.855059    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:26.961001 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:26.960968    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-9df039e4-866c7df7b7-56zff" event={"ID":"980f13c4-ba5f-4964-a3ef-2835f6916b99","Type":"ContainerStarted","Data":"ebbc7d0bf45c4d267b066e950e7864dd52914697befb975123a5d671373992d6"}
Oct 31 21:26:26.961272 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:26.961019    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-9df039e4-866c7df7b7-56zff" event={"ID":"980f13c4-ba5f-4964-a3ef-2835f6916b99","Type":"ContainerStarted","Data":"b1cdc81696a65692eafd1af017c28c72bfbb5299e763f25cdc1e8f34f2ab588f"}
Oct 31 21:26:26.961272 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:26.961257    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-9df039e4-866c7df7b7-56zff"
Oct 31 21:26:26.972570 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:26.972517    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:27.004766 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:27.004716    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-9df039e4-866c7df7b7-56zff" podStartSLOduration=2.004702642 podStartE2EDuration="2.004702642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:26:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:26:27.003857767 +0000 UTC m=+626.920167198" watchObservedRunningTime="2026-10-31 21:26:27.004702642 +0000 UTC m=+626.921012072"
Oct 31 21:26:27.489438 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:27.489401    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk"]
Oct 31 21:26:27.498076 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:27.498053    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk"
Oct 31 21:26:27.515889 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:27.515858    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk"]
Oct 31 21:26:27.521499 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:27.521477    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-25447e22-serving-cert\""
Oct 31 21:26:27.578681 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:27.578643    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:27.578823 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:27.578697    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bdd23d75-47de-4ce6-9ef0-6b418edc1336-metrics-certs\") pod \"maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk\" (UID: \"bdd23d75-47de-4ce6-9ef0-6b418edc1336\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk"
Oct 31 21:26:27.578823 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:27.578724    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bdd23d75-47de-4ce6-9ef0-6b418edc1336-maas-api-tls\") pod \"maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk\" (UID: \"bdd23d75-47de-4ce6-9ef0-6b418edc1336\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk"
Oct 31 21:26:27.578823 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:27.578816    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdd23d75-47de-4ce6-9ef0-6b418edc1336-openshift-service-ca\") pod \"maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk\" (UID: \"bdd23d75-47de-4ce6-9ef0-6b418edc1336\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk"
Oct 31 21:26:27.578961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:27.578836    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rggj\" (UniqueName: \"kubernetes.io/projected/bdd23d75-47de-4ce6-9ef0-6b418edc1336-kube-api-access-7rggj\") pod \"maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk\" (UID: \"bdd23d75-47de-4ce6-9ef0-6b418edc1336\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk"
Oct 31 21:26:27.679528 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:27.679500    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdd23d75-47de-4ce6-9ef0-6b418edc1336-openshift-service-ca\") pod \"maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk\" (UID: \"bdd23d75-47de-4ce6-9ef0-6b418edc1336\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk"
Oct 31 21:26:27.679528 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:27.679530    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rggj\" (UniqueName: \"kubernetes.io/projected/bdd23d75-47de-4ce6-9ef0-6b418edc1336-kube-api-access-7rggj\") pod \"maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk\" (UID: \"bdd23d75-47de-4ce6-9ef0-6b418edc1336\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk"
Oct 31 21:26:27.679729 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:27.679616    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bdd23d75-47de-4ce6-9ef0-6b418edc1336-metrics-certs\") pod \"maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk\" (UID: \"bdd23d75-47de-4ce6-9ef0-6b418edc1336\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk"
Oct 31 21:26:27.679729 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:27.679635    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bdd23d75-47de-4ce6-9ef0-6b418edc1336-maas-api-tls\") pod \"maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk\" (UID: \"bdd23d75-47de-4ce6-9ef0-6b418edc1336\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk"
Oct 31 21:26:27.680246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:27.680223    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdd23d75-47de-4ce6-9ef0-6b418edc1336-openshift-service-ca\") pod \"maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk\" (UID: \"bdd23d75-47de-4ce6-9ef0-6b418edc1336\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk"
Oct 31 21:26:27.681879 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:27.681859    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bdd23d75-47de-4ce6-9ef0-6b418edc1336-metrics-certs\") pod \"maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk\" (UID: \"bdd23d75-47de-4ce6-9ef0-6b418edc1336\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk"
Oct 31 21:26:27.681984 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:27.681901    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bdd23d75-47de-4ce6-9ef0-6b418edc1336-maas-api-tls\") pod \"maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk\" (UID: \"bdd23d75-47de-4ce6-9ef0-6b418edc1336\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk"
Oct 31 21:26:27.696862 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:27.696837    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:27.704251 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:27.704230    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rggj\" (UniqueName: \"kubernetes.io/projected/bdd23d75-47de-4ce6-9ef0-6b418edc1336-kube-api-access-7rggj\") pod \"maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk\" (UID: \"bdd23d75-47de-4ce6-9ef0-6b418edc1336\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk"
Oct 31 21:26:27.807080 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:27.807048    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:27.811922 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:27.811900    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk"
Oct 31 21:26:28.121620 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:28.121596    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk"]
Oct 31 21:26:28.125478 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:26:28.125438    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdd23d75_47de_4ce6_9ef0_6b418edc1336.slice/crio-d1d88ba2aa1e866e27c3d7dae2b19489575c298cd2c233ffdaf28a8ad44175a2 WatchSource:0}: Error finding container d1d88ba2aa1e866e27c3d7dae2b19489575c298cd2c233ffdaf28a8ad44175a2: Status 404 returned error can't find the container with id d1d88ba2aa1e866e27c3d7dae2b19489575c298cd2c233ffdaf28a8ad44175a2
Oct 31 21:26:28.977881 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:28.977847    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk" event={"ID":"bdd23d75-47de-4ce6-9ef0-6b418edc1336","Type":"ContainerStarted","Data":"a4906b5c5463dacc24cf26382ff8af986c1c2bd56a5ee3c4d0de074ec1507326"}
Oct 31 21:26:28.978068 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:28.977923    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk" event={"ID":"bdd23d75-47de-4ce6-9ef0-6b418edc1336","Type":"ContainerStarted","Data":"d1d88ba2aa1e866e27c3d7dae2b19489575c298cd2c233ffdaf28a8ad44175a2"}
Oct 31 21:26:28.978068 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:28.977965    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk"
Oct 31 21:26:29.027517 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:29.027475    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk" podStartSLOduration=1.673152182 podStartE2EDuration="2.027461698s" totalImagesPullingTime="354.309516ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:26:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:26:29.024464686 +0000 UTC m=+628.940774118" watchObservedRunningTime="2026-10-31 21:26:29.027461698 +0000 UTC m=+628.943771126"
Oct 31 21:26:29.226231 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:29.226192    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:29.354841 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:29.354807    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:29.481377 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:29.481342    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:29.597096 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:29.597038    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:31.360343 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:31.360306    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:31.966501 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:31.966475    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9df039e4-5f997854db-wbbnx"
Oct 31 21:26:32.945706 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:32.945668    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-353db9f5-69b57cffb7-6wt7s"
Oct 31 21:26:32.946178 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:32.945742    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-353db9f5-597d5d686c-xklsb"
Oct 31 21:26:33.145643 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:33.145605    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:33.948397 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:33.948361    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-25447e22-569c6647cf-ql62d"
Oct 31 21:26:33.948814 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:33.948449    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-25447e22-5998f5c689-sxqzp"
Oct 31 21:26:34.989992 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:34.989963    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk"
Oct 31 21:26:35.107459 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:35.107427    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:36.751811 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:36.751778    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:26:37.111112 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:37.111076    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:37.240705 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:37.240658    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:37.538587 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:37.538556    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:37.968803 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:37.968731    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-9df039e4-866c7df7b7-56zff"
Oct 31 21:26:39.054125 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:39.054076    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:41.048870 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:41.048834    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:42.068741 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:42.068706    4460 generic.go:356] "Generic (PLEG): container finished" podID="37956991-b640-4c37-acc6-888d3f4b1ef4" containerID="d9d1d5f30e9d671f43c60392e6ed5bcccc950cbd4c514577f184dad625ef6fe0" exitCode=0
Oct 31 21:26:42.069190 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:42.068746    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq" event={"ID":"37956991-b640-4c37-acc6-888d3f4b1ef4","Type":"ContainerDied","Data":"d9d1d5f30e9d671f43c60392e6ed5bcccc950cbd4c514577f184dad625ef6fe0"}
Oct 31 21:26:42.069190 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:42.069170    4460 scope.go:122] "RemoveContainer" containerID="d9d1d5f30e9d671f43c60392e6ed5bcccc950cbd4c514577f184dad625ef6fe0"
Oct 31 21:26:42.070419 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:42.070391    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:26:42.070478 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:42.070463    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:26:42.070533 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:42.070512    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:26:43.058964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:43.055779    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:43.076841 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:43.076804    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq" event={"ID":"37956991-b640-4c37-acc6-888d3f4b1ef4","Type":"ContainerStarted","Data":"c28fbda516829836facfc5f1ef1ca48ba83cc584e792166f0e2eb5a2b5dfe59e"}
Oct 31 21:26:43.444405 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:43.444311    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:43.444742 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:43.444655    4460 patch_prober.go:29] interesting pod/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 31 21:26:43.444902 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:43.444877    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq" podUID="37956991-b640-4c37-acc6-888d3f4b1ef4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 31 21:26:44.157473 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:44.157438    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:44.272185 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:44.272145    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:44.290217 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:44.290185    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-353db9f5-69b57cffb7-6wt7s"]
Oct 31 21:26:44.343966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:44.327670    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-353db9f5-597d5d686c-xklsb"]
Oct 31 21:26:44.433533 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:44.433444    4460 patch_prober.go:29] interesting pod/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 31 21:26:44.433799 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:44.433754    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq" podUID="37956991-b640-4c37-acc6-888d3f4b1ef4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 31 21:26:44.461133 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:44.461098    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:44.572678 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:44.572598    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:45.443964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:45.443805    4460 patch_prober.go:29] interesting pod/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 31 21:26:45.443964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:45.443872    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq" podUID="37956991-b640-4c37-acc6-888d3f4b1ef4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 31 21:26:45.445825 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:45.445789    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:45.517541 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:45.517512    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:26:46.442962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:46.442284    4460 patch_prober.go:29] interesting pod/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 31 21:26:46.442962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:46.442341    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq" podUID="37956991-b640-4c37-acc6-888d3f4b1ef4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 31 21:26:47.436849 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:47.436351    4460 patch_prober.go:29] interesting pod/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Oct 31 21:26:47.436849 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:47.436416    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq" podUID="37956991-b640-4c37-acc6-888d3f4b1ef4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Oct 31 21:26:47.688473 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:47.688371    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:47.905650 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:47.905615    4460 patch_prober.go:29] interesting pod/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 31 21:26:47.905809 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:47.905667    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2" podUID="75599b5e-0a6e-47e0-8598-39e58ee2edfd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 31 21:26:48.107523 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:48.107484    4460 generic.go:356] "Generic (PLEG): container finished" podID="75599b5e-0a6e-47e0-8598-39e58ee2edfd" containerID="8ae36d9c22de57b2c2af20f3e4b4dbe3117d6280358b8470a8fc0a8ce7a30502" exitCode=0
Oct 31 21:26:48.107727 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:48.107556    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2" event={"ID":"75599b5e-0a6e-47e0-8598-39e58ee2edfd","Type":"ContainerDied","Data":"8ae36d9c22de57b2c2af20f3e4b4dbe3117d6280358b8470a8fc0a8ce7a30502"}
Oct 31 21:26:48.108075 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:48.108057    4460 scope.go:122] "RemoveContainer" containerID="8ae36d9c22de57b2c2af20f3e4b4dbe3117d6280358b8470a8fc0a8ce7a30502"
Oct 31 21:26:48.109469 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:48.109437    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:26:48.109578 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:48.109533    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:26:48.109625 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:48.109588    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:26:48.432143 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:48.432070    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:48.432444 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:48.432411    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:48.433009 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:48.432989    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:26:49.113873 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:49.113835    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2" event={"ID":"75599b5e-0a6e-47e0-8598-39e58ee2edfd","Type":"ContainerStarted","Data":"cb7b38bb1aa10032caf4e17bb3604847a2cac10cd76af9838c5d773039aec454"}
Oct 31 21:26:49.290644 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:49.290600    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-353db9f5-69b57cffb7-6wt7s" podUID="2b2e0a10-c671-44b8-b748-9929dd416fe8" containerName="payload-processing" containerID="cri-o://0fe1f022ddceefd8b68b35b6f4cddd000459ba1aa9089dcd5f78b0f1d2e7bd23" gracePeriod=25
Oct 31 21:26:49.332966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:49.330023    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-353db9f5-597d5d686c-xklsb" podUID="8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa" containerName="payload-pre-processing" containerID="cri-o://b7522d6edf7b39c8f713b4f4536d1482b9e7356c1bf45583d038f811967f22a8" gracePeriod=25
Oct 31 21:26:49.347082 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:49.346816    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:49.611491 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:49.611462    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-353db9f5-597d5d686c-xklsb"
Oct 31 21:26:49.645645 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:49.645379    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:49.645645 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:49.645480    4460 patch_prober.go:29] interesting pod/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 31 21:26:49.645645 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:49.645523    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2" podUID="75599b5e-0a6e-47e0-8598-39e58ee2edfd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 31 21:26:49.651793 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:49.651772    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-353db9f5-69b57cffb7-6wt7s"
Oct 31 21:26:49.694801 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:49.694774    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa-plugins-config-volume\") pod \"8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa\" (UID: \"8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa\") "
Oct 31 21:26:49.694986 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:49.694815    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2b2e0a10-c671-44b8-b748-9929dd416fe8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b2e0a10-c671-44b8-b748-9929dd416fe8-plugins-config-volume\") pod \"2b2e0a10-c671-44b8-b748-9929dd416fe8\" (UID: \"2b2e0a10-c671-44b8-b748-9929dd416fe8\") "
Oct 31 21:26:49.694986 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:49.694855    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa-kube-api-access-gz5vb\" (UniqueName: \"kubernetes.io/projected/8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa-kube-api-access-gz5vb\") pod \"8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa\" (UID: \"8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa\") "
Oct 31 21:26:49.694986 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:49.694895    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b2e0a10-c671-44b8-b748-9929dd416fe8-kube-api-access-v5ssv\" (UniqueName: \"kubernetes.io/projected/2b2e0a10-c671-44b8-b748-9929dd416fe8-kube-api-access-v5ssv\") pod \"2b2e0a10-c671-44b8-b748-9929dd416fe8\" (UID: \"2b2e0a10-c671-44b8-b748-9929dd416fe8\") "
Oct 31 21:26:49.695256 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:49.695231    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa-plugins-config-volume" pod "8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa" (UID: "8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:26:49.695256 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:49.695236    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b2e0a10-c671-44b8-b748-9929dd416fe8-plugins-config-volume" pod "2b2e0a10-c671-44b8-b748-9929dd416fe8" (UID: "2b2e0a10-c671-44b8-b748-9929dd416fe8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:26:49.696876 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:49.696851    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b2e0a10-c671-44b8-b748-9929dd416fe8-kube-api-access-v5ssv" pod "2b2e0a10-c671-44b8-b748-9929dd416fe8" (UID: "2b2e0a10-c671-44b8-b748-9929dd416fe8"). InnerVolumeSpecName "kube-api-access-v5ssv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:26:49.697399 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:49.697383    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa-kube-api-access-gz5vb" pod "8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa" (UID: "8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa"). InnerVolumeSpecName "kube-api-access-gz5vb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:26:49.795795 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:49.795761    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:26:49.795795 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:49.795790    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b2e0a10-c671-44b8-b748-9929dd416fe8-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:26:49.795795 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:49.795801    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gz5vb\" (UniqueName: \"kubernetes.io/projected/8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa-kube-api-access-gz5vb\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:26:49.796040 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:49.795811    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5ssv\" (UniqueName: \"kubernetes.io/projected/2b2e0a10-c671-44b8-b748-9929dd416fe8-kube-api-access-v5ssv\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:26:50.119986 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:50.119876    4460 generic.go:356] "Generic (PLEG): container finished" podID="8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa" containerID="b7522d6edf7b39c8f713b4f4536d1482b9e7356c1bf45583d038f811967f22a8" exitCode=0
Oct 31 21:26:50.119986 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:50.119930    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-353db9f5-597d5d686c-xklsb"
Oct 31 21:26:50.120498 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:50.120022    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-353db9f5-597d5d686c-xklsb" event={"ID":"8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa","Type":"ContainerDied","Data":"b7522d6edf7b39c8f713b4f4536d1482b9e7356c1bf45583d038f811967f22a8"}
Oct 31 21:26:50.120498 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:50.120071    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-353db9f5-597d5d686c-xklsb" event={"ID":"8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa","Type":"ContainerDied","Data":"3bffaaa582f710b7523c83f2702e6533f559a162bdebc1a8ad534a8a38041f84"}
Oct 31 21:26:50.120498 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:50.120102    4460 scope.go:122] "RemoveContainer" containerID="b7522d6edf7b39c8f713b4f4536d1482b9e7356c1bf45583d038f811967f22a8"
Oct 31 21:26:50.146961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:50.144708    4460 generic.go:356] "Generic (PLEG): container finished" podID="2b2e0a10-c671-44b8-b748-9929dd416fe8" containerID="0fe1f022ddceefd8b68b35b6f4cddd000459ba1aa9089dcd5f78b0f1d2e7bd23" exitCode=0
Oct 31 21:26:50.146961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:50.145058    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-353db9f5-69b57cffb7-6wt7s" event={"ID":"2b2e0a10-c671-44b8-b748-9929dd416fe8","Type":"ContainerDied","Data":"0fe1f022ddceefd8b68b35b6f4cddd000459ba1aa9089dcd5f78b0f1d2e7bd23"}
Oct 31 21:26:50.146961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:50.145121    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-353db9f5-69b57cffb7-6wt7s" event={"ID":"2b2e0a10-c671-44b8-b748-9929dd416fe8","Type":"ContainerDied","Data":"1336a7a2a15d8d4bd5e5868bfc7c027cc66a172fab04a26016ed1f136d51ed0b"}
Oct 31 21:26:50.146961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:50.145228    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-353db9f5-69b57cffb7-6wt7s"
Oct 31 21:26:50.155698 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:50.155673    4460 scope.go:122] "RemoveContainer" containerID="b7522d6edf7b39c8f713b4f4536d1482b9e7356c1bf45583d038f811967f22a8"
Oct 31 21:26:50.156052 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:26:50.156032    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7522d6edf7b39c8f713b4f4536d1482b9e7356c1bf45583d038f811967f22a8\": container with ID starting with b7522d6edf7b39c8f713b4f4536d1482b9e7356c1bf45583d038f811967f22a8 not found: ID does not exist" containerID="b7522d6edf7b39c8f713b4f4536d1482b9e7356c1bf45583d038f811967f22a8"
Oct 31 21:26:50.156128 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:50.156063    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7522d6edf7b39c8f713b4f4536d1482b9e7356c1bf45583d038f811967f22a8"} err="failed to get container status \"b7522d6edf7b39c8f713b4f4536d1482b9e7356c1bf45583d038f811967f22a8\": rpc error: code = NotFound desc = could not find container \"b7522d6edf7b39c8f713b4f4536d1482b9e7356c1bf45583d038f811967f22a8\": container with ID starting with b7522d6edf7b39c8f713b4f4536d1482b9e7356c1bf45583d038f811967f22a8 not found: ID does not exist"
Oct 31 21:26:50.156128 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:50.156080    4460 scope.go:122] "RemoveContainer" containerID="0fe1f022ddceefd8b68b35b6f4cddd000459ba1aa9089dcd5f78b0f1d2e7bd23"
Oct 31 21:26:50.164121 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:50.164102    4460 scope.go:122] "RemoveContainer" containerID="0fe1f022ddceefd8b68b35b6f4cddd000459ba1aa9089dcd5f78b0f1d2e7bd23"
Oct 31 21:26:50.164413 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:26:50.164394    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fe1f022ddceefd8b68b35b6f4cddd000459ba1aa9089dcd5f78b0f1d2e7bd23\": container with ID starting with 0fe1f022ddceefd8b68b35b6f4cddd000459ba1aa9089dcd5f78b0f1d2e7bd23 not found: ID does not exist" containerID="0fe1f022ddceefd8b68b35b6f4cddd000459ba1aa9089dcd5f78b0f1d2e7bd23"
Oct 31 21:26:50.164479 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:50.164424    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fe1f022ddceefd8b68b35b6f4cddd000459ba1aa9089dcd5f78b0f1d2e7bd23"} err="failed to get container status \"0fe1f022ddceefd8b68b35b6f4cddd000459ba1aa9089dcd5f78b0f1d2e7bd23\": rpc error: code = NotFound desc = could not find container \"0fe1f022ddceefd8b68b35b6f4cddd000459ba1aa9089dcd5f78b0f1d2e7bd23\": container with ID starting with 0fe1f022ddceefd8b68b35b6f4cddd000459ba1aa9089dcd5f78b0f1d2e7bd23 not found: ID does not exist"
Oct 31 21:26:50.207468 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:50.207441    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-353db9f5-597d5d686c-xklsb"]
Oct 31 21:26:50.211716 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:50.211693    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-353db9f5-597d5d686c-xklsb"]
Oct 31 21:26:50.248201 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:50.248161    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa" path="/var/lib/kubelet/pods/8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa/volumes"
Oct 31 21:26:50.264890 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:50.264862    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-353db9f5-69b57cffb7-6wt7s"]
Oct 31 21:26:50.269159 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:50.269132    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-353db9f5-69b57cffb7-6wt7s"]
Oct 31 21:26:50.641963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:50.641301    4460 patch_prober.go:29] interesting pod/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 31 21:26:50.641963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:50.641380    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2" podUID="75599b5e-0a6e-47e0-8598-39e58ee2edfd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 31 21:26:51.146443 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:51.146405    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:51.468759 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:26:51.468677    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:26:51.643965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:51.643310    4460 patch_prober.go:29] interesting pod/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 31 21:26:51.643965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:51.643386    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2" podUID="75599b5e-0a6e-47e0-8598-39e58ee2edfd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 31 21:26:51.670037 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:51.669997    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9df039e4-866c7df7b7-56zff"]
Oct 31 21:26:51.779050 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:51.779011    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:51.892801 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:51.892769    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:52.246196 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:52.246113    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2b2e0a10-c671-44b8-b748-9929dd416fe8" path="/var/lib/kubelet/pods/2b2e0a10-c671-44b8-b748-9929dd416fe8/volumes"
Oct 31 21:26:52.645326 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:52.645291    4460 patch_prober.go:29] interesting pod/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 31 21:26:52.645501 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:52.645352    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2" podUID="75599b5e-0a6e-47e0-8598-39e58ee2edfd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 31 21:26:53.081177 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:53.081137    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:53.641963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:53.641260    4460 patch_prober.go:29] interesting pod/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Oct 31 21:26:53.641963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:53.641322    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2" podUID="75599b5e-0a6e-47e0-8598-39e58ee2edfd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Oct 31 21:26:54.645996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:54.645967    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:54.646391 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:54.646352    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:54.647009 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:54.646990    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:26:54.719312 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:54.719275    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:55.004976 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:55.004878    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9df039e4-5f997854db-wbbnx"]
Oct 31 21:26:55.005333 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:55.005304    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9df039e4-5f997854db-wbbnx" podUID="f94f57bd-1c2d-420b-ac9f-0806a79335e0" containerName="maas-api" containerID="cri-o://23c4b717584a910177a9c81a3562321d06ed11f9f95fb350e3c599a7ab3af8a5" gracePeriod=30
Oct 31 21:26:55.073030 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:55.073001    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:55.176876 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:55.176844    4460 generic.go:356] "Generic (PLEG): container finished" podID="f94f57bd-1c2d-420b-ac9f-0806a79335e0" containerID="23c4b717584a910177a9c81a3562321d06ed11f9f95fb350e3c599a7ab3af8a5" exitCode=0
Oct 31 21:26:55.177054 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:55.176918    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9df039e4-5f997854db-wbbnx" event={"ID":"f94f57bd-1c2d-420b-ac9f-0806a79335e0","Type":"ContainerDied","Data":"23c4b717584a910177a9c81a3562321d06ed11f9f95fb350e3c599a7ab3af8a5"}
Oct 31 21:26:55.202016 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:55.201980    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:55.264877 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:55.264855    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9df039e4-5f997854db-wbbnx"
Oct 31 21:26:55.324106 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:55.324063    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:55.372862 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:55.372831    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f94f57bd-1c2d-420b-ac9f-0806a79335e0-kube-api-access-q7ndq\" (UniqueName: \"kubernetes.io/projected/f94f57bd-1c2d-420b-ac9f-0806a79335e0-kube-api-access-q7ndq\") pod \"f94f57bd-1c2d-420b-ac9f-0806a79335e0\" (UID: \"f94f57bd-1c2d-420b-ac9f-0806a79335e0\") "
Oct 31 21:26:55.373055 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:55.372963    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f94f57bd-1c2d-420b-ac9f-0806a79335e0-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f94f57bd-1c2d-420b-ac9f-0806a79335e0-maas-api-tls\") pod \"f94f57bd-1c2d-420b-ac9f-0806a79335e0\" (UID: \"f94f57bd-1c2d-420b-ac9f-0806a79335e0\") "
Oct 31 21:26:55.373055 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:55.373004    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f94f57bd-1c2d-420b-ac9f-0806a79335e0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f94f57bd-1c2d-420b-ac9f-0806a79335e0-metrics-certs\") pod \"f94f57bd-1c2d-420b-ac9f-0806a79335e0\" (UID: \"f94f57bd-1c2d-420b-ac9f-0806a79335e0\") "
Oct 31 21:26:55.373055 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:55.373051    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f94f57bd-1c2d-420b-ac9f-0806a79335e0-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f94f57bd-1c2d-420b-ac9f-0806a79335e0-openshift-service-ca\") pod \"f94f57bd-1c2d-420b-ac9f-0806a79335e0\" (UID: \"f94f57bd-1c2d-420b-ac9f-0806a79335e0\") "
Oct 31 21:26:55.373453 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:55.373425    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f94f57bd-1c2d-420b-ac9f-0806a79335e0-openshift-service-ca" pod "f94f57bd-1c2d-420b-ac9f-0806a79335e0" (UID: "f94f57bd-1c2d-420b-ac9f-0806a79335e0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:26:55.375033 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:55.375009    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f94f57bd-1c2d-420b-ac9f-0806a79335e0-kube-api-access-q7ndq" pod "f94f57bd-1c2d-420b-ac9f-0806a79335e0" (UID: "f94f57bd-1c2d-420b-ac9f-0806a79335e0"). InnerVolumeSpecName "kube-api-access-q7ndq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:26:55.375033 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:55.375014    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f94f57bd-1c2d-420b-ac9f-0806a79335e0-maas-api-tls" pod "f94f57bd-1c2d-420b-ac9f-0806a79335e0" (UID: "f94f57bd-1c2d-420b-ac9f-0806a79335e0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 21:26:55.375171 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:55.375098    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f94f57bd-1c2d-420b-ac9f-0806a79335e0-metrics-certs" pod "f94f57bd-1c2d-420b-ac9f-0806a79335e0" (UID: "f94f57bd-1c2d-420b-ac9f-0806a79335e0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 21:26:55.474315 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:55.474282    4460 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f94f57bd-1c2d-420b-ac9f-0806a79335e0-openshift-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:26:55.474315 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:55.474315    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7ndq\" (UniqueName: \"kubernetes.io/projected/f94f57bd-1c2d-420b-ac9f-0806a79335e0-kube-api-access-q7ndq\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:26:55.474543 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:55.474330    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f94f57bd-1c2d-420b-ac9f-0806a79335e0-maas-api-tls\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:26:55.474543 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:55.474344    4460 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f94f57bd-1c2d-420b-ac9f-0806a79335e0-metrics-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:26:55.491357 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:55.491311    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:55.618970 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:55.618867    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:55.824342 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:55.824295    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:55.951438 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:55.951336    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:56.075161 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.075110    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:56.159175 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.159137    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-25447e22-5998f5c689-sxqzp"]
Oct 31 21:26:56.185171 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.185140    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9df039e4-5f997854db-wbbnx"
Oct 31 21:26:56.185357 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.185142    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-9df039e4-5f997854db-wbbnx" event={"ID":"f94f57bd-1c2d-420b-ac9f-0806a79335e0","Type":"ContainerDied","Data":"93b992c90732a79580655b9a812e37ce966dcaf9ea5eb952f20c5f02056e0bd4"}
Oct 31 21:26:56.185357 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.185287    4460 scope.go:122] "RemoveContainer" containerID="23c4b717584a910177a9c81a3562321d06ed11f9f95fb350e3c599a7ab3af8a5"
Oct 31 21:26:56.198767 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.198735    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-25447e22-569c6647cf-ql62d"]
Oct 31 21:26:56.248898 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.248863    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9df039e4-5f997854db-wbbnx"]
Oct 31 21:26:56.253131 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.253103    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-9df039e4-5f997854db-wbbnx"]
Oct 31 21:26:56.308920 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.308880    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:56.427417 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.427344    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:56.671332 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.671294    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-9df039e4-866c7df7b7-56zff" podUID="980f13c4-ba5f-4964-a3ef-2835f6916b99" containerName="payload-pre-processing" containerID="cri-o://ebbc7d0bf45c4d267b066e950e7864dd52914697befb975123a5d671373992d6" gracePeriod=25
Oct 31 21:26:56.740262 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.740231    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"]
Oct 31 21:26:56.740477 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.740455    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq" podUID="37956991-b640-4c37-acc6-888d3f4b1ef4" containerName="istio-proxy" containerID="cri-o://c28fbda516829836facfc5f1ef1ca48ba83cc584e792166f0e2eb5a2b5dfe59e" gracePeriod=30
Oct 31 21:26:56.777223 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.777190    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:56.833216 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.833182    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"]
Oct 31 21:26:56.833605 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.833592    4460 cpu_manager.go:414] "containerMap: removing container" podUID="2b2e0a10-c671-44b8-b748-9929dd416fe8" containerName="payload-processing"
Oct 31 21:26:56.833642 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.833606    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b2e0a10-c671-44b8-b748-9929dd416fe8" containerName="payload-processing"
Oct 31 21:26:56.833642 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.833629    4460 cpu_manager.go:414] "containerMap: removing container" podUID="f94f57bd-1c2d-420b-ac9f-0806a79335e0" containerName="maas-api"
Oct 31 21:26:56.833642 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.833634    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f94f57bd-1c2d-420b-ac9f-0806a79335e0" containerName="maas-api"
Oct 31 21:26:56.833733 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.833655    4460 cpu_manager.go:414] "containerMap: removing container" podUID="8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa" containerName="payload-pre-processing"
Oct 31 21:26:56.833733 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.833661    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa" containerName="payload-pre-processing"
Oct 31 21:26:56.833733 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.833720    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="8f44fe4f-9c3c-4519-bcb1-3f21a1c6a8aa" containerName="payload-pre-processing"
Oct 31 21:26:56.833733 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.833728    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b2e0a10-c671-44b8-b748-9929dd416fe8" containerName="payload-processing"
Oct 31 21:26:56.833848 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.833737    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="f94f57bd-1c2d-420b-ac9f-0806a79335e0" containerName="maas-api"
Oct 31 21:26:56.836982 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.836966    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:26:56.854527 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.854499    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-106555-openshift-default-dockercfg-v5bxg\""
Oct 31 21:26:56.862361 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.862337    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"]
Oct 31 21:26:56.887158 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.887132    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea04c855-12eb-4628-a947-3b4e8321bef1-istio-podinfo\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:26:56.887328 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.887166    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfjgw\" (UniqueName: \"kubernetes.io/projected/ea04c855-12eb-4628-a947-3b4e8321bef1-kube-api-access-cfjgw\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:26:56.887328 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.887190    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea04c855-12eb-4628-a947-3b4e8321bef1-istio-token\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:26:56.887328 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.887243    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-workload-socket\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:26:56.887501 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.887360    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-credential-socket\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:26:56.887501 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.887381    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-istio-data\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:26:56.887501 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.887424    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-istio-envoy\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:26:56.887501 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.887454    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-workload-certs\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:26:56.887679 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.887542    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea04c855-12eb-4628-a947-3b4e8321bef1-istiod-ca-cert\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:26:56.910469 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.910437    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:56.969046 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.969019    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-9df039e4-866c7df7b7-56zff"
Oct 31 21:26:56.988161 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.988133    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/980f13c4-ba5f-4964-a3ef-2835f6916b99-kube-api-access-m4d6l\" (UniqueName: \"kubernetes.io/projected/980f13c4-ba5f-4964-a3ef-2835f6916b99-kube-api-access-m4d6l\") pod \"980f13c4-ba5f-4964-a3ef-2835f6916b99\" (UID: \"980f13c4-ba5f-4964-a3ef-2835f6916b99\") "
Oct 31 21:26:56.988328 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.988219    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/980f13c4-ba5f-4964-a3ef-2835f6916b99-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/980f13c4-ba5f-4964-a3ef-2835f6916b99-plugins-config-volume\") pod \"980f13c4-ba5f-4964-a3ef-2835f6916b99\" (UID: \"980f13c4-ba5f-4964-a3ef-2835f6916b99\") "
Oct 31 21:26:56.988447 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.988429    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfjgw\" (UniqueName: \"kubernetes.io/projected/ea04c855-12eb-4628-a947-3b4e8321bef1-kube-api-access-cfjgw\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:26:56.988515 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.988464    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea04c855-12eb-4628-a947-3b4e8321bef1-istio-token\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:26:56.988515 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.988493    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-workload-socket\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:26:56.988623 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.988558    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-credential-socket\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:26:56.988623 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.988582    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-istio-data\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:26:56.988623 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.988612    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-istio-envoy\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:26:56.988770 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.988646    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/980f13c4-ba5f-4964-a3ef-2835f6916b99-plugins-config-volume" pod "980f13c4-ba5f-4964-a3ef-2835f6916b99" (UID: "980f13c4-ba5f-4964-a3ef-2835f6916b99"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:26:56.988770 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.988657    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-workload-certs\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:26:56.988770 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.988735    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea04c855-12eb-4628-a947-3b4e8321bef1-istiod-ca-cert\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:26:56.988918 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.988858    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea04c855-12eb-4628-a947-3b4e8321bef1-istio-podinfo\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:26:56.989022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.988919    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/980f13c4-ba5f-4964-a3ef-2835f6916b99-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:26:56.989022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.988997    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-workload-certs\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:26:56.989133 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.989042    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-credential-socket\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:26:56.989133 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.989060    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-istio-data\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:26:56.989360 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.989339    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-workload-socket\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:26:56.989607 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.989580    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea04c855-12eb-4628-a947-3b4e8321bef1-istiod-ca-cert\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:26:56.990614 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.990595    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/980f13c4-ba5f-4964-a3ef-2835f6916b99-kube-api-access-m4d6l" pod "980f13c4-ba5f-4964-a3ef-2835f6916b99" (UID: "980f13c4-ba5f-4964-a3ef-2835f6916b99"). InnerVolumeSpecName "kube-api-access-m4d6l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:26:56.991239 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.991203    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-istio-envoy\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:26:56.991393 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:56.991371    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea04c855-12eb-4628-a947-3b4e8321bef1-istio-podinfo\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:26:57.013586 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:57.013568    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea04c855-12eb-4628-a947-3b4e8321bef1-istio-token\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:26:57.013675 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:57.013590    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfjgw\" (UniqueName: \"kubernetes.io/projected/ea04c855-12eb-4628-a947-3b4e8321bef1-kube-api-access-cfjgw\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:26:57.089803 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:57.089771    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4d6l\" (UniqueName: \"kubernetes.io/projected/980f13c4-ba5f-4964-a3ef-2835f6916b99-kube-api-access-m4d6l\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:26:57.109281 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:57.109253    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:57.152834 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:57.152804    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:26:57.194577 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:57.194476    4460 generic.go:356] "Generic (PLEG): container finished" podID="980f13c4-ba5f-4964-a3ef-2835f6916b99" containerID="ebbc7d0bf45c4d267b066e950e7864dd52914697befb975123a5d671373992d6" exitCode=0
Oct 31 21:26:57.194577 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:57.194541    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-9df039e4-866c7df7b7-56zff"
Oct 31 21:26:57.194577 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:57.194550    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-9df039e4-866c7df7b7-56zff" event={"ID":"980f13c4-ba5f-4964-a3ef-2835f6916b99","Type":"ContainerDied","Data":"ebbc7d0bf45c4d267b066e950e7864dd52914697befb975123a5d671373992d6"}
Oct 31 21:26:57.194783 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:57.194607    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-9df039e4-866c7df7b7-56zff" event={"ID":"980f13c4-ba5f-4964-a3ef-2835f6916b99","Type":"ContainerDied","Data":"b1cdc81696a65692eafd1af017c28c72bfbb5299e763f25cdc1e8f34f2ab588f"}
Oct 31 21:26:57.194783 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:57.194626    4460 scope.go:122] "RemoveContainer" containerID="ebbc7d0bf45c4d267b066e950e7864dd52914697befb975123a5d671373992d6"
Oct 31 21:26:57.203702 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:57.203679    4460 scope.go:122] "RemoveContainer" containerID="ebbc7d0bf45c4d267b066e950e7864dd52914697befb975123a5d671373992d6"
Oct 31 21:26:57.204099 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:26:57.204038    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebbc7d0bf45c4d267b066e950e7864dd52914697befb975123a5d671373992d6\": container with ID starting with ebbc7d0bf45c4d267b066e950e7864dd52914697befb975123a5d671373992d6 not found: ID does not exist" containerID="ebbc7d0bf45c4d267b066e950e7864dd52914697befb975123a5d671373992d6"
Oct 31 21:26:57.204099 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:57.204082    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebbc7d0bf45c4d267b066e950e7864dd52914697befb975123a5d671373992d6"} err="failed to get container status \"ebbc7d0bf45c4d267b066e950e7864dd52914697befb975123a5d671373992d6\": rpc error: code = NotFound desc = could not find container \"ebbc7d0bf45c4d267b066e950e7864dd52914697befb975123a5d671373992d6\": container with ID starting with ebbc7d0bf45c4d267b066e950e7864dd52914697befb975123a5d671373992d6 not found: ID does not exist"
Oct 31 21:26:57.250915 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:57.250879    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9df039e4-866c7df7b7-56zff"]
Oct 31 21:26:57.255332 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:57.255298    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-9df039e4-866c7df7b7-56zff"]
Oct 31 21:26:57.474400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:57.474370    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"]
Oct 31 21:26:57.476426 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:26:57.476401    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea04c855_12eb_4628_a947_3b4e8321bef1.slice/crio-adb80e1f73b32803b706087594a9301b0d5b96769d514e54222d0daf6731bd63 WatchSource:0}: Error finding container adb80e1f73b32803b706087594a9301b0d5b96769d514e54222d0daf6731bd63: Status 404 returned error can't find the container with id adb80e1f73b32803b706087594a9301b0d5b96769d514e54222d0daf6731bd63
Oct 31 21:26:57.478600 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:57.478574    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:26:57.478643 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:57.478632    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:26:57.478683 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:57.478663    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:26:57.518424 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:57.518396    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:57.933698 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:57.933664    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"]
Oct 31 21:26:57.934336 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:57.934315    4460 cpu_manager.go:414] "containerMap: removing container" podUID="980f13c4-ba5f-4964-a3ef-2835f6916b99" containerName="payload-pre-processing"
Oct 31 21:26:57.934336 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:57.934336    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="980f13c4-ba5f-4964-a3ef-2835f6916b99" containerName="payload-pre-processing"
Oct 31 21:26:57.934502 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:57.934481    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="980f13c4-ba5f-4964-a3ef-2835f6916b99" containerName="payload-pre-processing"
Oct 31 21:26:57.937744 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:57.937724    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:26:57.952734 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:57.947980    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:57.961829 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:57.961807    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"]
Oct 31 21:26:58.000017 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:57.999985    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9501cf33-637a-4d00-8e58-3ae157dd6840-istio-token\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:26:58.000017 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.000015    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjfwb\" (UniqueName: \"kubernetes.io/projected/9501cf33-637a-4d00-8e58-3ae157dd6840-kube-api-access-mjfwb\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:26:58.000229 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.000036    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-credential-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:26:58.000266 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.000218    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9501cf33-637a-4d00-8e58-3ae157dd6840-istiod-ca-cert\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:26:58.000266 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.000259    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-workload-certs\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:26:58.000328 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.000316    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9501cf33-637a-4d00-8e58-3ae157dd6840-istio-podinfo\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:26:58.000375 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.000344    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-istio-data\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:26:58.000453 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.000435    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-istio-envoy\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:26:58.000516 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.000501    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-workload-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:26:58.001019 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.001000    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-4b5989-openshift-default-dockercfg-gpzdz\""
Oct 31 21:26:58.064874 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.064835    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:58.102178 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.102127    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-istio-envoy\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:26:58.102352 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.102214    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-workload-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:26:58.102352 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.102247    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9501cf33-637a-4d00-8e58-3ae157dd6840-istio-token\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:26:58.102352 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.102272    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjfwb\" (UniqueName: \"kubernetes.io/projected/9501cf33-637a-4d00-8e58-3ae157dd6840-kube-api-access-mjfwb\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:26:58.102352 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.102303    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-credential-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:26:58.102569 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.102405    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9501cf33-637a-4d00-8e58-3ae157dd6840-istiod-ca-cert\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:26:58.102569 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.102443    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-workload-certs\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:26:58.102569 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.102474    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9501cf33-637a-4d00-8e58-3ae157dd6840-istio-podinfo\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:26:58.102569 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.102506    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-istio-data\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:26:58.102988 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.102961    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-istio-data\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:26:58.103192 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.103089    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-workload-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:26:58.104302 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.104276    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-credential-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:26:58.105379 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.104988    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9501cf33-637a-4d00-8e58-3ae157dd6840-istiod-ca-cert\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:26:58.105379 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.105232    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-workload-certs\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:26:58.138386 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.138350    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9501cf33-637a-4d00-8e58-3ae157dd6840-istio-podinfo\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:26:58.138641 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.138506    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-istio-envoy\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:26:58.141717 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.141678    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjfwb\" (UniqueName: \"kubernetes.io/projected/9501cf33-637a-4d00-8e58-3ae157dd6840-kube-api-access-mjfwb\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:26:58.142300 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.142263    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9501cf33-637a-4d00-8e58-3ae157dd6840-istio-token\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:26:58.176895 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.176850    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:58.201799 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.201716    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z" event={"ID":"ea04c855-12eb-4628-a947-3b4e8321bef1","Type":"ContainerStarted","Data":"9cd7432e1359bfff5b62fbe8a1c9facffb89c649f1993ea889ced7ada81fd433"}
Oct 31 21:26:58.201799 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.201793    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z" event={"ID":"ea04c855-12eb-4628-a947-3b4e8321bef1","Type":"ContainerStarted","Data":"adb80e1f73b32803b706087594a9301b0d5b96769d514e54222d0daf6731bd63"}
Oct 31 21:26:58.222792 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.222761    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="980f13c4-ba5f-4964-a3ef-2835f6916b99" path="/var/lib/kubelet/pods/980f13c4-ba5f-4964-a3ef-2835f6916b99/volumes"
Oct 31 21:26:58.223308 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.223291    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f94f57bd-1c2d-420b-ac9f-0806a79335e0" path="/var/lib/kubelet/pods/f94f57bd-1c2d-420b-ac9f-0806a79335e0/volumes"
Oct 31 21:26:58.253358 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.253316    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z" podStartSLOduration=2.253303521 podStartE2EDuration="2.253303521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:26:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:26:58.252805664 +0000 UTC m=+658.169115093" watchObservedRunningTime="2026-10-31 21:26:58.253303521 +0000 UTC m=+658.169612950"
Oct 31 21:26:58.254928 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.254910    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:26:58.575275 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.575249    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"]
Oct 31 21:26:58.577174 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:26:58.577148    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9501cf33_637a_4d00_8e58_3ae157dd6840.slice/crio-66e9431332d3ad3d5a00200b4317b20d291ece95b5cf10536b148dc8e9b3b4d3 WatchSource:0}: Error finding container 66e9431332d3ad3d5a00200b4317b20d291ece95b5cf10536b148dc8e9b3b4d3: Status 404 returned error can't find the container with id 66e9431332d3ad3d5a00200b4317b20d291ece95b5cf10536b148dc8e9b3b4d3
Oct 31 21:26:58.579373 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.579345    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:26:58.579443 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.579406    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:26:58.579443 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.579434    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:26:58.662483 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:58.662448    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:26:59.153917 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:59.153886    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:26:59.158841 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:59.158817    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:26:59.208745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:59.208710    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9" event={"ID":"9501cf33-637a-4d00-8e58-3ae157dd6840","Type":"ContainerStarted","Data":"e2c939512586f906bab898a18a9dabd22b7a1e5615e0e076315b70cc71573229"}
Oct 31 21:26:59.208871 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:59.208766    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9" event={"ID":"9501cf33-637a-4d00-8e58-3ae157dd6840","Type":"ContainerStarted","Data":"66e9431332d3ad3d5a00200b4317b20d291ece95b5cf10536b148dc8e9b3b4d3"}
Oct 31 21:26:59.208971 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:59.208955    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:26:59.209705 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:59.209686    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:26:59.255994 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:59.255972    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:26:59.261017 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:59.260998    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:26:59.318417 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:59.318369    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9" podStartSLOduration=2.318355022 podStartE2EDuration="2.318355022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:26:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:26:59.316780406 +0000 UTC m=+659.233089835" watchObservedRunningTime="2026-10-31 21:26:59.318355022 +0000 UTC m=+659.234664450"
Oct 31 21:26:59.500698 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:59.500622    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk"]
Oct 31 21:26:59.500840 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:59.500825    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk" podUID="bdd23d75-47de-4ce6-9ef0-6b418edc1336" containerName="maas-api" containerID="cri-o://a4906b5c5463dacc24cf26382ff8af986c1c2bd56a5ee3c4d0de074ec1507326" gracePeriod=30
Oct 31 21:26:59.595926 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:59.595887    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:26:59.745202 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:59.745180    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk"
Oct 31 21:26:59.822220 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:59.822196    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bdd23d75-47de-4ce6-9ef0-6b418edc1336-kube-api-access-7rggj\" (UniqueName: \"kubernetes.io/projected/bdd23d75-47de-4ce6-9ef0-6b418edc1336-kube-api-access-7rggj\") pod \"bdd23d75-47de-4ce6-9ef0-6b418edc1336\" (UID: \"bdd23d75-47de-4ce6-9ef0-6b418edc1336\") "
Oct 31 21:26:59.822356 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:59.822288    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bdd23d75-47de-4ce6-9ef0-6b418edc1336-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bdd23d75-47de-4ce6-9ef0-6b418edc1336-maas-api-tls\") pod \"bdd23d75-47de-4ce6-9ef0-6b418edc1336\" (UID: \"bdd23d75-47de-4ce6-9ef0-6b418edc1336\") "
Oct 31 21:26:59.822409 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:59.822389    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bdd23d75-47de-4ce6-9ef0-6b418edc1336-metrics-certs\" (UniqueName: \"kubernetes.io/secret/bdd23d75-47de-4ce6-9ef0-6b418edc1336-metrics-certs\") pod \"bdd23d75-47de-4ce6-9ef0-6b418edc1336\" (UID: \"bdd23d75-47de-4ce6-9ef0-6b418edc1336\") "
Oct 31 21:26:59.822450 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:59.822436    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bdd23d75-47de-4ce6-9ef0-6b418edc1336-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdd23d75-47de-4ce6-9ef0-6b418edc1336-openshift-service-ca\") pod \"bdd23d75-47de-4ce6-9ef0-6b418edc1336\" (UID: \"bdd23d75-47de-4ce6-9ef0-6b418edc1336\") "
Oct 31 21:26:59.822899 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:59.822871    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdd23d75-47de-4ce6-9ef0-6b418edc1336-openshift-service-ca" pod "bdd23d75-47de-4ce6-9ef0-6b418edc1336" (UID: "bdd23d75-47de-4ce6-9ef0-6b418edc1336"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:26:59.824357 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:59.824336    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bdd23d75-47de-4ce6-9ef0-6b418edc1336-metrics-certs" pod "bdd23d75-47de-4ce6-9ef0-6b418edc1336" (UID: "bdd23d75-47de-4ce6-9ef0-6b418edc1336"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 21:26:59.824422 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:59.824394    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bdd23d75-47de-4ce6-9ef0-6b418edc1336-maas-api-tls" pod "bdd23d75-47de-4ce6-9ef0-6b418edc1336" (UID: "bdd23d75-47de-4ce6-9ef0-6b418edc1336"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 21:26:59.824422 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:59.824400    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdd23d75-47de-4ce6-9ef0-6b418edc1336-kube-api-access-7rggj" pod "bdd23d75-47de-4ce6-9ef0-6b418edc1336" (UID: "bdd23d75-47de-4ce6-9ef0-6b418edc1336"). InnerVolumeSpecName "kube-api-access-7rggj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:26:59.923349 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:59.923324    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bdd23d75-47de-4ce6-9ef0-6b418edc1336-maas-api-tls\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:26:59.923349 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:59.923346    4460 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bdd23d75-47de-4ce6-9ef0-6b418edc1336-metrics-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:26:59.923477 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:59.923357    4460 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdd23d75-47de-4ce6-9ef0-6b418edc1336-openshift-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:26:59.923477 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:26:59.923366    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rggj\" (UniqueName: \"kubernetes.io/projected/bdd23d75-47de-4ce6-9ef0-6b418edc1336-kube-api-access-7rggj\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:27:00.119226 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:00.119146    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:00.215075 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:00.215032    4460 generic.go:356] "Generic (PLEG): container finished" podID="bdd23d75-47de-4ce6-9ef0-6b418edc1336" containerID="a4906b5c5463dacc24cf26382ff8af986c1c2bd56a5ee3c4d0de074ec1507326" exitCode=0
Oct 31 21:27:00.215509 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:00.215073    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk" event={"ID":"bdd23d75-47de-4ce6-9ef0-6b418edc1336","Type":"ContainerDied","Data":"a4906b5c5463dacc24cf26382ff8af986c1c2bd56a5ee3c4d0de074ec1507326"}
Oct 31 21:27:00.215509 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:00.215127    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk" event={"ID":"bdd23d75-47de-4ce6-9ef0-6b418edc1336","Type":"ContainerDied","Data":"d1d88ba2aa1e866e27c3d7dae2b19489575c298cd2c233ffdaf28a8ad44175a2"}
Oct 31 21:27:00.215509 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:00.215089    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk"
Oct 31 21:27:00.215509 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:00.215144    4460 scope.go:122] "RemoveContainer" containerID="a4906b5c5463dacc24cf26382ff8af986c1c2bd56a5ee3c4d0de074ec1507326"
Oct 31 21:27:00.215509 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:00.215339    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:27:00.216642 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:00.216622    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:27:00.225444 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:00.225424    4460 scope.go:122] "RemoveContainer" containerID="a4906b5c5463dacc24cf26382ff8af986c1c2bd56a5ee3c4d0de074ec1507326"
Oct 31 21:27:00.225719 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:27:00.225701    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4906b5c5463dacc24cf26382ff8af986c1c2bd56a5ee3c4d0de074ec1507326\": container with ID starting with a4906b5c5463dacc24cf26382ff8af986c1c2bd56a5ee3c4d0de074ec1507326 not found: ID does not exist" containerID="a4906b5c5463dacc24cf26382ff8af986c1c2bd56a5ee3c4d0de074ec1507326"
Oct 31 21:27:00.225779 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:00.225729    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4906b5c5463dacc24cf26382ff8af986c1c2bd56a5ee3c4d0de074ec1507326"} err="failed to get container status \"a4906b5c5463dacc24cf26382ff8af986c1c2bd56a5ee3c4d0de074ec1507326\": rpc error: code = NotFound desc = could not find container \"a4906b5c5463dacc24cf26382ff8af986c1c2bd56a5ee3c4d0de074ec1507326\": container with ID starting with a4906b5c5463dacc24cf26382ff8af986c1c2bd56a5ee3c4d0de074ec1507326 not found: ID does not exist"
Oct 31 21:27:00.248615 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:00.248596    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:00.324014 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:00.323979    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk"]
Oct 31 21:27:00.329932 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:00.329905    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-25447e22-c585d4fcc-ckxnk"]
Oct 31 21:27:00.364764 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:00.364737    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:00.693477 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:00.693443    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:00.821763 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:00.821640    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:00.979445 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:00.979370    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"]
Oct 31 21:27:00.979605 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:00.979369    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:00.979680 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:00.979609    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2" podUID="75599b5e-0a6e-47e0-8598-39e58ee2edfd" containerName="istio-proxy" containerID="cri-o://cb7b38bb1aa10032caf4e17bb3604847a2cac10cd76af9838c5d773039aec454" gracePeriod=30
Oct 31 21:27:01.099757 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:01.099727    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:01.160335 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:01.160282    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-25447e22-5998f5c689-sxqzp" podUID="fd4777f2-6998-443b-b61c-ce07542d48f4" containerName="payload-processing" containerID="cri-o://e3105f6450be35591d53a2bc68cce19897365ae1cd37e93687fdd50d80144060" gracePeriod=25
Oct 31 21:27:01.199616 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:01.199583    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-25447e22-569c6647cf-ql62d" podUID="8875cc55-d9a9-4759-add2-2ce8d59a4a02" containerName="payload-pre-processing" containerID="cri-o://6709243c32a3c5f3a2bb5615300fd42cc5547482c7548aedb166a98465faabed" gracePeriod=25
Oct 31 21:27:01.441635 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:01.441612    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-25447e22-5998f5c689-sxqzp"
Oct 31 21:27:01.467563 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:01.467541    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-25447e22-569c6647cf-ql62d"
Oct 31 21:27:01.539245 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:01.539207    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd4777f2-6998-443b-b61c-ce07542d48f4-kube-api-access-mgn97\" (UniqueName: \"kubernetes.io/projected/fd4777f2-6998-443b-b61c-ce07542d48f4-kube-api-access-mgn97\") pod \"fd4777f2-6998-443b-b61c-ce07542d48f4\" (UID: \"fd4777f2-6998-443b-b61c-ce07542d48f4\") "
Oct 31 21:27:01.539419 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:01.539265    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fd4777f2-6998-443b-b61c-ce07542d48f4-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd4777f2-6998-443b-b61c-ce07542d48f4-plugins-config-volume\") pod \"fd4777f2-6998-443b-b61c-ce07542d48f4\" (UID: \"fd4777f2-6998-443b-b61c-ce07542d48f4\") "
Oct 31 21:27:01.539419 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:01.539378    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8875cc55-d9a9-4759-add2-2ce8d59a4a02-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8875cc55-d9a9-4759-add2-2ce8d59a4a02-plugins-config-volume\") pod \"8875cc55-d9a9-4759-add2-2ce8d59a4a02\" (UID: \"8875cc55-d9a9-4759-add2-2ce8d59a4a02\") "
Oct 31 21:27:01.539540 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:01.539457    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8875cc55-d9a9-4759-add2-2ce8d59a4a02-kube-api-access-wndcx\" (UniqueName: \"kubernetes.io/projected/8875cc55-d9a9-4759-add2-2ce8d59a4a02-kube-api-access-wndcx\") pod \"8875cc55-d9a9-4759-add2-2ce8d59a4a02\" (UID: \"8875cc55-d9a9-4759-add2-2ce8d59a4a02\") "
Oct 31 21:27:01.539654 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:01.539634    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd4777f2-6998-443b-b61c-ce07542d48f4-plugins-config-volume" pod "fd4777f2-6998-443b-b61c-ce07542d48f4" (UID: "fd4777f2-6998-443b-b61c-ce07542d48f4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:27:01.539769 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:01.539742    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8875cc55-d9a9-4759-add2-2ce8d59a4a02-plugins-config-volume" pod "8875cc55-d9a9-4759-add2-2ce8d59a4a02" (UID: "8875cc55-d9a9-4759-add2-2ce8d59a4a02"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:27:01.539896 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:01.539882    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd4777f2-6998-443b-b61c-ce07542d48f4-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:27:01.539983 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:01.539901    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8875cc55-d9a9-4759-add2-2ce8d59a4a02-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:27:01.541596 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:01.541574    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8875cc55-d9a9-4759-add2-2ce8d59a4a02-kube-api-access-wndcx" pod "8875cc55-d9a9-4759-add2-2ce8d59a4a02" (UID: "8875cc55-d9a9-4759-add2-2ce8d59a4a02"). InnerVolumeSpecName "kube-api-access-wndcx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:27:01.541827 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:01.541808    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd4777f2-6998-443b-b61c-ce07542d48f4-kube-api-access-mgn97" pod "fd4777f2-6998-443b-b61c-ce07542d48f4" (UID: "fd4777f2-6998-443b-b61c-ce07542d48f4"). InnerVolumeSpecName "kube-api-access-mgn97". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:27:01.641023 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:01.640996    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mgn97\" (UniqueName: \"kubernetes.io/projected/fd4777f2-6998-443b-b61c-ce07542d48f4-kube-api-access-mgn97\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:27:01.641023 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:01.641019    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wndcx\" (UniqueName: \"kubernetes.io/projected/8875cc55-d9a9-4759-add2-2ce8d59a4a02-kube-api-access-wndcx\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:27:01.837490 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:01.837406    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:01.952859 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:01.952824    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:01.980674 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:01.980653    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:27:02.044662 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.044627    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-workload-certs\") pod \"37956991-b640-4c37-acc6-888d3f4b1ef4\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") "
Oct 31 21:27:02.044878 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.044771    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/37956991-b640-4c37-acc6-888d3f4b1ef4-istio-token\" (UniqueName: \"kubernetes.io/projected/37956991-b640-4c37-acc6-888d3f4b1ef4-istio-token\") pod \"37956991-b640-4c37-acc6-888d3f4b1ef4\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") "
Oct 31 21:27:02.044878 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.044804    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/37956991-b640-4c37-acc6-888d3f4b1ef4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37956991-b640-4c37-acc6-888d3f4b1ef4-istiod-ca-cert\") pod \"37956991-b640-4c37-acc6-888d3f4b1ef4\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") "
Oct 31 21:27:02.044878 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.044840    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-istio-data\") pod \"37956991-b640-4c37-acc6-888d3f4b1ef4\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") "
Oct 31 21:27:02.044878 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.044871    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/37956991-b640-4c37-acc6-888d3f4b1ef4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37956991-b640-4c37-acc6-888d3f4b1ef4-istio-podinfo\") pod \"37956991-b640-4c37-acc6-888d3f4b1ef4\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") "
Oct 31 21:27:02.045088 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.044902    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/37956991-b640-4c37-acc6-888d3f4b1ef4-kube-api-access-vjkgm\" (UniqueName: \"kubernetes.io/projected/37956991-b640-4c37-acc6-888d3f4b1ef4-kube-api-access-vjkgm\") pod \"37956991-b640-4c37-acc6-888d3f4b1ef4\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") "
Oct 31 21:27:02.045088 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.044917    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-workload-certs" pod "37956991-b640-4c37-acc6-888d3f4b1ef4" (UID: "37956991-b640-4c37-acc6-888d3f4b1ef4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:27:02.045088 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.044930    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-istio-envoy\") pod \"37956991-b640-4c37-acc6-888d3f4b1ef4\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") "
Oct 31 21:27:02.045088 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.045026    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-credential-socket\") pod \"37956991-b640-4c37-acc6-888d3f4b1ef4\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") "
Oct 31 21:27:02.045221 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.045087    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-workload-socket\") pod \"37956991-b640-4c37-acc6-888d3f4b1ef4\" (UID: \"37956991-b640-4c37-acc6-888d3f4b1ef4\") "
Oct 31 21:27:02.045221 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.045098    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-istio-data" pod "37956991-b640-4c37-acc6-888d3f4b1ef4" (UID: "37956991-b640-4c37-acc6-888d3f4b1ef4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:27:02.045569 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.045470    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:27:02.045569 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.045498    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:27:02.045781 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.045758    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-workload-socket" pod "37956991-b640-4c37-acc6-888d3f4b1ef4" (UID: "37956991-b640-4c37-acc6-888d3f4b1ef4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:27:02.045845 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.045686    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37956991-b640-4c37-acc6-888d3f4b1ef4-istiod-ca-cert" pod "37956991-b640-4c37-acc6-888d3f4b1ef4" (UID: "37956991-b640-4c37-acc6-888d3f4b1ef4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:27:02.046022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.045995    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-credential-socket" pod "37956991-b640-4c37-acc6-888d3f4b1ef4" (UID: "37956991-b640-4c37-acc6-888d3f4b1ef4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:27:02.048818 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.048791    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/37956991-b640-4c37-acc6-888d3f4b1ef4-istio-podinfo" pod "37956991-b640-4c37-acc6-888d3f4b1ef4" (UID: "37956991-b640-4c37-acc6-888d3f4b1ef4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:27:02.050486 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.050460    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-istio-envoy" pod "37956991-b640-4c37-acc6-888d3f4b1ef4" (UID: "37956991-b640-4c37-acc6-888d3f4b1ef4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:27:02.050727 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.050697    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37956991-b640-4c37-acc6-888d3f4b1ef4-kube-api-access-vjkgm" pod "37956991-b640-4c37-acc6-888d3f4b1ef4" (UID: "37956991-b640-4c37-acc6-888d3f4b1ef4"). InnerVolumeSpecName "kube-api-access-vjkgm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:27:02.052074 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.052055    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37956991-b640-4c37-acc6-888d3f4b1ef4-istio-token" pod "37956991-b640-4c37-acc6-888d3f4b1ef4" (UID: "37956991-b640-4c37-acc6-888d3f4b1ef4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:27:02.146856 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.146780    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37956991-b640-4c37-acc6-888d3f4b1ef4-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:27:02.146856 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.146810    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37956991-b640-4c37-acc6-888d3f4b1ef4-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:27:02.146856 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.146819    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37956991-b640-4c37-acc6-888d3f4b1ef4-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:27:02.146856 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.146827    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjkgm\" (UniqueName: \"kubernetes.io/projected/37956991-b640-4c37-acc6-888d3f4b1ef4-kube-api-access-vjkgm\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:27:02.146856 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.146837    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:27:02.146856 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.146845    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:27:02.146856 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.146854    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37956991-b640-4c37-acc6-888d3f4b1ef4-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:27:02.223583 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.223550    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bdd23d75-47de-4ce6-9ef0-6b418edc1336" path="/var/lib/kubelet/pods/bdd23d75-47de-4ce6-9ef0-6b418edc1336/volumes"
Oct 31 21:27:02.226022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.225994    4460 generic.go:356] "Generic (PLEG): container finished" podID="37956991-b640-4c37-acc6-888d3f4b1ef4" containerID="c28fbda516829836facfc5f1ef1ca48ba83cc584e792166f0e2eb5a2b5dfe59e" exitCode=0
Oct 31 21:27:02.226187 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.226042    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"
Oct 31 21:27:02.226187 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.226082    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq" event={"ID":"37956991-b640-4c37-acc6-888d3f4b1ef4","Type":"ContainerDied","Data":"c28fbda516829836facfc5f1ef1ca48ba83cc584e792166f0e2eb5a2b5dfe59e"}
Oct 31 21:27:02.226187 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.226136    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq" event={"ID":"37956991-b640-4c37-acc6-888d3f4b1ef4","Type":"ContainerDied","Data":"df35047c31baf4a60a2006d4496e67868a7551e878c06d4dff835d9514636a23"}
Oct 31 21:27:02.226187 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.226163    4460 scope.go:122] "RemoveContainer" containerID="c28fbda516829836facfc5f1ef1ca48ba83cc584e792166f0e2eb5a2b5dfe59e"
Oct 31 21:27:02.229177 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.229152    4460 generic.go:356] "Generic (PLEG): container finished" podID="8875cc55-d9a9-4759-add2-2ce8d59a4a02" containerID="6709243c32a3c5f3a2bb5615300fd42cc5547482c7548aedb166a98465faabed" exitCode=0
Oct 31 21:27:02.229502 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.229235    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-25447e22-569c6647cf-ql62d"
Oct 31 21:27:02.229502 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.229233    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-25447e22-569c6647cf-ql62d" event={"ID":"8875cc55-d9a9-4759-add2-2ce8d59a4a02","Type":"ContainerDied","Data":"6709243c32a3c5f3a2bb5615300fd42cc5547482c7548aedb166a98465faabed"}
Oct 31 21:27:02.229502 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.229408    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-25447e22-569c6647cf-ql62d" event={"ID":"8875cc55-d9a9-4759-add2-2ce8d59a4a02","Type":"ContainerDied","Data":"347298440c16e2fbbf4be1c10cddc0eec55f828fc53e8dde89f54d7915f70db0"}
Oct 31 21:27:02.234617 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.231123    4460 generic.go:356] "Generic (PLEG): container finished" podID="fd4777f2-6998-443b-b61c-ce07542d48f4" containerID="e3105f6450be35591d53a2bc68cce19897365ae1cd37e93687fdd50d80144060" exitCode=0
Oct 31 21:27:02.234617 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.231212    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-25447e22-5998f5c689-sxqzp"
Oct 31 21:27:02.234617 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.231242    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-25447e22-5998f5c689-sxqzp" event={"ID":"fd4777f2-6998-443b-b61c-ce07542d48f4","Type":"ContainerDied","Data":"e3105f6450be35591d53a2bc68cce19897365ae1cd37e93687fdd50d80144060"}
Oct 31 21:27:02.234617 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.231278    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-25447e22-5998f5c689-sxqzp" event={"ID":"fd4777f2-6998-443b-b61c-ce07542d48f4","Type":"ContainerDied","Data":"9acae83c548d653d62563279f28f4f39c0afe00cf92028ea65dfa5656746e95f"}
Oct 31 21:27:02.237585 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.237570    4460 scope.go:122] "RemoveContainer" containerID="d9d1d5f30e9d671f43c60392e6ed5bcccc950cbd4c514577f184dad625ef6fe0"
Oct 31 21:27:02.246674 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.246658    4460 scope.go:122] "RemoveContainer" containerID="c28fbda516829836facfc5f1ef1ca48ba83cc584e792166f0e2eb5a2b5dfe59e"
Oct 31 21:27:02.246927 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:27:02.246909    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c28fbda516829836facfc5f1ef1ca48ba83cc584e792166f0e2eb5a2b5dfe59e\": container with ID starting with c28fbda516829836facfc5f1ef1ca48ba83cc584e792166f0e2eb5a2b5dfe59e not found: ID does not exist" containerID="c28fbda516829836facfc5f1ef1ca48ba83cc584e792166f0e2eb5a2b5dfe59e"
Oct 31 21:27:02.247025 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.246935    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c28fbda516829836facfc5f1ef1ca48ba83cc584e792166f0e2eb5a2b5dfe59e"} err="failed to get container status \"c28fbda516829836facfc5f1ef1ca48ba83cc584e792166f0e2eb5a2b5dfe59e\": rpc error: code = NotFound desc = could not find container \"c28fbda516829836facfc5f1ef1ca48ba83cc584e792166f0e2eb5a2b5dfe59e\": container with ID starting with c28fbda516829836facfc5f1ef1ca48ba83cc584e792166f0e2eb5a2b5dfe59e not found: ID does not exist"
Oct 31 21:27:02.247025 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.246969    4460 scope.go:122] "RemoveContainer" containerID="d9d1d5f30e9d671f43c60392e6ed5bcccc950cbd4c514577f184dad625ef6fe0"
Oct 31 21:27:02.247215 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:27:02.247200    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9d1d5f30e9d671f43c60392e6ed5bcccc950cbd4c514577f184dad625ef6fe0\": container with ID starting with d9d1d5f30e9d671f43c60392e6ed5bcccc950cbd4c514577f184dad625ef6fe0 not found: ID does not exist" containerID="d9d1d5f30e9d671f43c60392e6ed5bcccc950cbd4c514577f184dad625ef6fe0"
Oct 31 21:27:02.247256 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.247219    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9d1d5f30e9d671f43c60392e6ed5bcccc950cbd4c514577f184dad625ef6fe0"} err="failed to get container status \"d9d1d5f30e9d671f43c60392e6ed5bcccc950cbd4c514577f184dad625ef6fe0\": rpc error: code = NotFound desc = could not find container \"d9d1d5f30e9d671f43c60392e6ed5bcccc950cbd4c514577f184dad625ef6fe0\": container with ID starting with d9d1d5f30e9d671f43c60392e6ed5bcccc950cbd4c514577f184dad625ef6fe0 not found: ID does not exist"
Oct 31 21:27:02.247256 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.247232    4460 scope.go:122] "RemoveContainer" containerID="6709243c32a3c5f3a2bb5615300fd42cc5547482c7548aedb166a98465faabed"
Oct 31 21:27:02.256285 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.256262    4460 scope.go:122] "RemoveContainer" containerID="6709243c32a3c5f3a2bb5615300fd42cc5547482c7548aedb166a98465faabed"
Oct 31 21:27:02.256625 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:27:02.256601    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6709243c32a3c5f3a2bb5615300fd42cc5547482c7548aedb166a98465faabed\": container with ID starting with 6709243c32a3c5f3a2bb5615300fd42cc5547482c7548aedb166a98465faabed not found: ID does not exist" containerID="6709243c32a3c5f3a2bb5615300fd42cc5547482c7548aedb166a98465faabed"
Oct 31 21:27:02.256721 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.256636    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6709243c32a3c5f3a2bb5615300fd42cc5547482c7548aedb166a98465faabed"} err="failed to get container status \"6709243c32a3c5f3a2bb5615300fd42cc5547482c7548aedb166a98465faabed\": rpc error: code = NotFound desc = could not find container \"6709243c32a3c5f3a2bb5615300fd42cc5547482c7548aedb166a98465faabed\": container with ID starting with 6709243c32a3c5f3a2bb5615300fd42cc5547482c7548aedb166a98465faabed not found: ID does not exist"
Oct 31 21:27:02.256721 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.256659    4460 scope.go:122] "RemoveContainer" containerID="e3105f6450be35591d53a2bc68cce19897365ae1cd37e93687fdd50d80144060"
Oct 31 21:27:02.268617 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.268596    4460 scope.go:122] "RemoveContainer" containerID="e3105f6450be35591d53a2bc68cce19897365ae1cd37e93687fdd50d80144060"
Oct 31 21:27:02.268981 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:27:02.268956    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3105f6450be35591d53a2bc68cce19897365ae1cd37e93687fdd50d80144060\": container with ID starting with e3105f6450be35591d53a2bc68cce19897365ae1cd37e93687fdd50d80144060 not found: ID does not exist" containerID="e3105f6450be35591d53a2bc68cce19897365ae1cd37e93687fdd50d80144060"
Oct 31 21:27:02.269073 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.268988    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3105f6450be35591d53a2bc68cce19897365ae1cd37e93687fdd50d80144060"} err="failed to get container status \"e3105f6450be35591d53a2bc68cce19897365ae1cd37e93687fdd50d80144060\": rpc error: code = NotFound desc = could not find container \"e3105f6450be35591d53a2bc68cce19897365ae1cd37e93687fdd50d80144060\": container with ID starting with e3105f6450be35591d53a2bc68cce19897365ae1cd37e93687fdd50d80144060 not found: ID does not exist"
Oct 31 21:27:02.284054 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.284033    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-25447e22-5998f5c689-sxqzp"]
Oct 31 21:27:02.288871 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.288849    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-25447e22-5998f5c689-sxqzp"]
Oct 31 21:27:02.324550 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.324521    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:02.344784 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.344757    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"]
Oct 31 21:27:02.349696 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.349673    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-9df039e4-openshift-default-7c9847cc65-xmntq"]
Oct 31 21:27:02.422423 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.422346    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-25447e22-569c6647cf-ql62d"]
Oct 31 21:27:02.434811 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.434782    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-25447e22-569c6647cf-ql62d"]
Oct 31 21:27:02.596628 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.596596    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:02.717591 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.717511    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:02.845186 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.845152    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:02.981934 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:02.981848    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:03.085801 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.085766    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:27:03.313149 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.313117    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:03.318002 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.317976    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn"]
Oct 31 21:27:03.318371 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.318359    4460 cpu_manager.go:414] "containerMap: removing container" podUID="bdd23d75-47de-4ce6-9ef0-6b418edc1336" containerName="maas-api"
Oct 31 21:27:03.318423 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.318372    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bdd23d75-47de-4ce6-9ef0-6b418edc1336" containerName="maas-api"
Oct 31 21:27:03.318423 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.318384    4460 cpu_manager.go:414] "containerMap: removing container" podUID="37956991-b640-4c37-acc6-888d3f4b1ef4" containerName="istio-proxy"
Oct 31 21:27:03.318423 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.318389    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="37956991-b640-4c37-acc6-888d3f4b1ef4" containerName="istio-proxy"
Oct 31 21:27:03.318423 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.318406    4460 cpu_manager.go:414] "containerMap: removing container" podUID="fd4777f2-6998-443b-b61c-ce07542d48f4" containerName="payload-processing"
Oct 31 21:27:03.318423 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.318411    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fd4777f2-6998-443b-b61c-ce07542d48f4" containerName="payload-processing"
Oct 31 21:27:03.318423 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.318421    4460 cpu_manager.go:414] "containerMap: removing container" podUID="37956991-b640-4c37-acc6-888d3f4b1ef4" containerName="istio-proxy"
Oct 31 21:27:03.318423 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.318426    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="37956991-b640-4c37-acc6-888d3f4b1ef4" containerName="istio-proxy"
Oct 31 21:27:03.318649 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.318448    4460 cpu_manager.go:414] "containerMap: removing container" podUID="8875cc55-d9a9-4759-add2-2ce8d59a4a02" containerName="payload-pre-processing"
Oct 31 21:27:03.318649 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.318452    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8875cc55-d9a9-4759-add2-2ce8d59a4a02" containerName="payload-pre-processing"
Oct 31 21:27:03.318649 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.318503    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="8875cc55-d9a9-4759-add2-2ce8d59a4a02" containerName="payload-pre-processing"
Oct 31 21:27:03.318649 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.318511    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="bdd23d75-47de-4ce6-9ef0-6b418edc1336" containerName="maas-api"
Oct 31 21:27:03.318649 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.318516    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="37956991-b640-4c37-acc6-888d3f4b1ef4" containerName="istio-proxy"
Oct 31 21:27:03.318649 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.318532    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="fd4777f2-6998-443b-b61c-ce07542d48f4" containerName="payload-processing"
Oct 31 21:27:03.318649 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.318539    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="37956991-b640-4c37-acc6-888d3f4b1ef4" containerName="istio-proxy"
Oct 31 21:27:03.323324 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.323305    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn"
Oct 31 21:27:03.345386 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.345363    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn"]
Oct 31 21:27:03.404711 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.381246    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-106555-serving-cert\""
Oct 31 21:27:03.447817 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.447783    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:03.542965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.539711    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ee0ad1f-c884-4875-8af0-b494cf74c092-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn\" (UID: \"8ee0ad1f-c884-4875-8af0-b494cf74c092\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn"
Oct 31 21:27:03.542965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.539883    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ee0ad1f-c884-4875-8af0-b494cf74c092-metrics-certs\") pod \"maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn\" (UID: \"8ee0ad1f-c884-4875-8af0-b494cf74c092\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn"
Oct 31 21:27:03.542965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.539917    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pjk5\" (UniqueName: \"kubernetes.io/projected/8ee0ad1f-c884-4875-8af0-b494cf74c092-kube-api-access-9pjk5\") pod \"maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn\" (UID: \"8ee0ad1f-c884-4875-8af0-b494cf74c092\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn"
Oct 31 21:27:03.542965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.540028    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ee0ad1f-c884-4875-8af0-b494cf74c092-maas-api-tls\") pod \"maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn\" (UID: \"8ee0ad1f-c884-4875-8af0-b494cf74c092\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn"
Oct 31 21:27:03.570393 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.570299    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:03.640558 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.640522    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ee0ad1f-c884-4875-8af0-b494cf74c092-maas-api-tls\") pod \"maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn\" (UID: \"8ee0ad1f-c884-4875-8af0-b494cf74c092\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn"
Oct 31 21:27:03.641053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.640586    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ee0ad1f-c884-4875-8af0-b494cf74c092-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn\" (UID: \"8ee0ad1f-c884-4875-8af0-b494cf74c092\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn"
Oct 31 21:27:03.641053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.640678    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ee0ad1f-c884-4875-8af0-b494cf74c092-metrics-certs\") pod \"maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn\" (UID: \"8ee0ad1f-c884-4875-8af0-b494cf74c092\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn"
Oct 31 21:27:03.641053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.640710    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pjk5\" (UniqueName: \"kubernetes.io/projected/8ee0ad1f-c884-4875-8af0-b494cf74c092-kube-api-access-9pjk5\") pod \"maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn\" (UID: \"8ee0ad1f-c884-4875-8af0-b494cf74c092\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn"
Oct 31 21:27:03.641529 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.641499    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ee0ad1f-c884-4875-8af0-b494cf74c092-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn\" (UID: \"8ee0ad1f-c884-4875-8af0-b494cf74c092\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn"
Oct 31 21:27:03.643034 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.643009    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ee0ad1f-c884-4875-8af0-b494cf74c092-metrics-certs\") pod \"maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn\" (UID: \"8ee0ad1f-c884-4875-8af0-b494cf74c092\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn"
Oct 31 21:27:03.643146 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.643044    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ee0ad1f-c884-4875-8af0-b494cf74c092-maas-api-tls\") pod \"maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn\" (UID: \"8ee0ad1f-c884-4875-8af0-b494cf74c092\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn"
Oct 31 21:27:03.666030 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.665995    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pjk5\" (UniqueName: \"kubernetes.io/projected/8ee0ad1f-c884-4875-8af0-b494cf74c092-kube-api-access-9pjk5\") pod \"maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn\" (UID: \"8ee0ad1f-c884-4875-8af0-b494cf74c092\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn"
Oct 31 21:27:03.683019 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.682982    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:03.803009 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.802975    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:03.924876 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.924795    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:03.935689 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.935660    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn"
Oct 31 21:27:03.990295 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.990212    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-106555-659c88c8f5-7hdjm"]
Oct 31 21:27:04.000803 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:03.996055    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-106555-659c88c8f5-7hdjm"
Oct 31 21:27:04.017460 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.017408    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-106555\""
Oct 31 21:27:04.017643 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.017546    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-106555-dockercfg-9hprd\""
Oct 31 21:27:04.017783 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.017743    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-106555-659c88c8f5-7hdjm"]
Oct 31 21:27:04.059144 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.054822    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v58mz\" (UniqueName: \"kubernetes.io/projected/9fb4edcd-09a0-4f56-9391-574b046c514e-kube-api-access-v58mz\") pod \"payload-pre-processing-e2e-worker-w3-106555-659c88c8f5-7hdjm\" (UID: \"9fb4edcd-09a0-4f56-9391-574b046c514e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-106555-659c88c8f5-7hdjm"
Oct 31 21:27:04.059144 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.054923    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fb4edcd-09a0-4f56-9391-574b046c514e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-106555-659c88c8f5-7hdjm\" (UID: \"9fb4edcd-09a0-4f56-9391-574b046c514e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-106555-659c88c8f5-7hdjm"
Oct 31 21:27:04.078247 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.077873    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:04.115374 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.115336    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-106555-849b8bdbd5-4trlx"]
Oct 31 21:27:04.119653 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.119632    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-106555-849b8bdbd5-4trlx"]
Oct 31 21:27:04.119871 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.119758    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-106555-849b8bdbd5-4trlx"
Oct 31 21:27:04.155403 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.155368    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84jrn\" (UniqueName: \"kubernetes.io/projected/342810a1-1edf-4dde-ac1d-b8844e81d5ac-kube-api-access-84jrn\") pod \"payload-processing-e2e-worker-w3-106555-849b8bdbd5-4trlx\" (UID: \"342810a1-1edf-4dde-ac1d-b8844e81d5ac\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-106555-849b8bdbd5-4trlx"
Oct 31 21:27:04.155526 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.155416    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fb4edcd-09a0-4f56-9391-574b046c514e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-106555-659c88c8f5-7hdjm\" (UID: \"9fb4edcd-09a0-4f56-9391-574b046c514e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-106555-659c88c8f5-7hdjm"
Oct 31 21:27:04.155601 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.155582    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/342810a1-1edf-4dde-ac1d-b8844e81d5ac-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-106555-849b8bdbd5-4trlx\" (UID: \"342810a1-1edf-4dde-ac1d-b8844e81d5ac\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-106555-849b8bdbd5-4trlx"
Oct 31 21:27:04.155659 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.155628    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v58mz\" (UniqueName: \"kubernetes.io/projected/9fb4edcd-09a0-4f56-9391-574b046c514e-kube-api-access-v58mz\") pod \"payload-pre-processing-e2e-worker-w3-106555-659c88c8f5-7hdjm\" (UID: \"9fb4edcd-09a0-4f56-9391-574b046c514e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-106555-659c88c8f5-7hdjm"
Oct 31 21:27:04.156089 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.156067    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fb4edcd-09a0-4f56-9391-574b046c514e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-106555-659c88c8f5-7hdjm\" (UID: \"9fb4edcd-09a0-4f56-9391-574b046c514e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-106555-659c88c8f5-7hdjm"
Oct 31 21:27:04.182036 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.181975    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v58mz\" (UniqueName: \"kubernetes.io/projected/9fb4edcd-09a0-4f56-9391-574b046c514e-kube-api-access-v58mz\") pod \"payload-pre-processing-e2e-worker-w3-106555-659c88c8f5-7hdjm\" (UID: \"9fb4edcd-09a0-4f56-9391-574b046c514e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-106555-659c88c8f5-7hdjm"
Oct 31 21:27:04.227722 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.227686    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="37956991-b640-4c37-acc6-888d3f4b1ef4" path="/var/lib/kubelet/pods/37956991-b640-4c37-acc6-888d3f4b1ef4/volumes"
Oct 31 21:27:04.228891 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.228867    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8875cc55-d9a9-4759-add2-2ce8d59a4a02" path="/var/lib/kubelet/pods/8875cc55-d9a9-4759-add2-2ce8d59a4a02/volumes"
Oct 31 21:27:04.229660 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.229641    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fd4777f2-6998-443b-b61c-ce07542d48f4" path="/var/lib/kubelet/pods/fd4777f2-6998-443b-b61c-ce07542d48f4/volumes"
Oct 31 21:27:04.245849 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.245825    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn"]
Oct 31 21:27:04.256428 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.256405    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84jrn\" (UniqueName: \"kubernetes.io/projected/342810a1-1edf-4dde-ac1d-b8844e81d5ac-kube-api-access-84jrn\") pod \"payload-processing-e2e-worker-w3-106555-849b8bdbd5-4trlx\" (UID: \"342810a1-1edf-4dde-ac1d-b8844e81d5ac\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-106555-849b8bdbd5-4trlx"
Oct 31 21:27:04.256521 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.256502    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/342810a1-1edf-4dde-ac1d-b8844e81d5ac-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-106555-849b8bdbd5-4trlx\" (UID: \"342810a1-1edf-4dde-ac1d-b8844e81d5ac\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-106555-849b8bdbd5-4trlx"
Oct 31 21:27:04.257124 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.257105    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/342810a1-1edf-4dde-ac1d-b8844e81d5ac-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-106555-849b8bdbd5-4trlx\" (UID: \"342810a1-1edf-4dde-ac1d-b8844e81d5ac\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-106555-849b8bdbd5-4trlx"
Oct 31 21:27:04.281259 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.281235    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84jrn\" (UniqueName: \"kubernetes.io/projected/342810a1-1edf-4dde-ac1d-b8844e81d5ac-kube-api-access-84jrn\") pod \"payload-processing-e2e-worker-w3-106555-849b8bdbd5-4trlx\" (UID: \"342810a1-1edf-4dde-ac1d-b8844e81d5ac\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-106555-849b8bdbd5-4trlx"
Oct 31 21:27:04.318224 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.318200    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-106555-659c88c8f5-7hdjm"
Oct 31 21:27:04.434343 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.434277    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-106555-849b8bdbd5-4trlx"
Oct 31 21:27:04.617038 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.617017    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-106555-659c88c8f5-7hdjm"]
Oct 31 21:27:04.734686 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.734650    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-106555-849b8bdbd5-4trlx"]
Oct 31 21:27:04.792034 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.792001    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:04.906808 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.906769    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:04.950224 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.950154    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-4b5989-7bdb89b88c-qgl9q"]
Oct 31 21:27:04.953622 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.953606    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b5989-7bdb89b88c-qgl9q"
Oct 31 21:27:04.963090 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.963066    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pmpd\" (UniqueName: \"kubernetes.io/projected/4c072735-3c6f-4c6a-9d51-b535fe7f4dfe-kube-api-access-4pmpd\") pod \"payload-pre-processing-e2e-worker-w0-4b5989-7bdb89b88c-qgl9q\" (UID: \"4c072735-3c6f-4c6a-9d51-b535fe7f4dfe\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b5989-7bdb89b88c-qgl9q"
Oct 31 21:27:04.963221 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.963202    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c072735-3c6f-4c6a-9d51-b535fe7f4dfe-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-4b5989-7bdb89b88c-qgl9q\" (UID: \"4c072735-3c6f-4c6a-9d51-b535fe7f4dfe\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b5989-7bdb89b88c-qgl9q"
Oct 31 21:27:04.974448 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.974425    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-4b5989-dockercfg-7rdqm\""
Oct 31 21:27:04.974559 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.974501    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-4b5989\""
Oct 31 21:27:04.976705 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:04.976682    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-4b5989-7bdb89b88c-qgl9q"]
Oct 31 21:27:05.064920 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:05.064153    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pmpd\" (UniqueName: \"kubernetes.io/projected/4c072735-3c6f-4c6a-9d51-b535fe7f4dfe-kube-api-access-4pmpd\") pod \"payload-pre-processing-e2e-worker-w0-4b5989-7bdb89b88c-qgl9q\" (UID: \"4c072735-3c6f-4c6a-9d51-b535fe7f4dfe\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b5989-7bdb89b88c-qgl9q"
Oct 31 21:27:05.065972 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:05.065902    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c072735-3c6f-4c6a-9d51-b535fe7f4dfe-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-4b5989-7bdb89b88c-qgl9q\" (UID: \"4c072735-3c6f-4c6a-9d51-b535fe7f4dfe\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b5989-7bdb89b88c-qgl9q"
Oct 31 21:27:05.069489 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:05.068901    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c072735-3c6f-4c6a-9d51-b535fe7f4dfe-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-4b5989-7bdb89b88c-qgl9q\" (UID: \"4c072735-3c6f-4c6a-9d51-b535fe7f4dfe\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b5989-7bdb89b88c-qgl9q"
Oct 31 21:27:05.101696 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:05.101660    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pmpd\" (UniqueName: \"kubernetes.io/projected/4c072735-3c6f-4c6a-9d51-b535fe7f4dfe-kube-api-access-4pmpd\") pod \"payload-pre-processing-e2e-worker-w0-4b5989-7bdb89b88c-qgl9q\" (UID: \"4c072735-3c6f-4c6a-9d51-b535fe7f4dfe\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b5989-7bdb89b88c-qgl9q"
Oct 31 21:27:05.255244 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:05.255074    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-106555-659c88c8f5-7hdjm" event={"ID":"9fb4edcd-09a0-4f56-9391-574b046c514e","Type":"ContainerStarted","Data":"1c841413ed6dc40751b20a30e77ce57fe9da641439459fcedb8b39acbe5a3b11"}
Oct 31 21:27:05.255244 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:05.255159    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-106555-659c88c8f5-7hdjm"
Oct 31 21:27:05.255244 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:05.255183    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-106555-659c88c8f5-7hdjm" event={"ID":"9fb4edcd-09a0-4f56-9391-574b046c514e","Type":"ContainerStarted","Data":"13b73916edaa4ba98555b4a9a67e8b7929655fbf73d70d7b1c8e8c4b9ef597d1"}
Oct 31 21:27:05.256845 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:05.256816    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn" event={"ID":"8ee0ad1f-c884-4875-8af0-b494cf74c092","Type":"ContainerStarted","Data":"0b6bd0423aa1d0e19c220da129343ef6b2a71b4c2646631230ddcff22680b950"}
Oct 31 21:27:05.256980 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:05.256863    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn" event={"ID":"8ee0ad1f-c884-4875-8af0-b494cf74c092","Type":"ContainerStarted","Data":"8de53173304a411e97c2083b85ecda8a715f7e5c812bad671cc614853c8e7434"}
Oct 31 21:27:05.256980 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:05.256929    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn"
Oct 31 21:27:05.258505 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:05.258482    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-106555-849b8bdbd5-4trlx" event={"ID":"342810a1-1edf-4dde-ac1d-b8844e81d5ac","Type":"ContainerStarted","Data":"39c1609bab709638d5b200c461eeeaf52d3c142cb7c0b35e8b8ed4de76847b38"}
Oct 31 21:27:05.258607 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:05.258517    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-106555-849b8bdbd5-4trlx" event={"ID":"342810a1-1edf-4dde-ac1d-b8844e81d5ac","Type":"ContainerStarted","Data":"605b4528c0d48953915555c851b8fda6425fb5fffbd4cb35341c30ada4f6de06"}
Oct 31 21:27:05.258607 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:05.258561    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-106555-849b8bdbd5-4trlx"
Oct 31 21:27:05.266192 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:05.266174    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b5989-7bdb89b88c-qgl9q"
Oct 31 21:27:05.301061 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:05.300687    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:05.302281 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:05.302227    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-106555-659c88c8f5-7hdjm" podStartSLOduration=2.302207983 podStartE2EDuration="2.302207983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:27:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:27:05.299528814 +0000 UTC m=+665.215838244" watchObservedRunningTime="2026-10-31 21:27:05.302207983 +0000 UTC m=+665.218517411"
Oct 31 21:27:05.348708 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:05.348656    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn" podStartSLOduration=2.020573423 podStartE2EDuration="2.348641286s" totalImagesPullingTime="328.067863ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:27:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:27:05.346471298 +0000 UTC m=+665.262780726" watchObservedRunningTime="2026-10-31 21:27:05.348641286 +0000 UTC m=+665.264950714"
Oct 31 21:27:05.397371 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:05.397325    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w3-106555-849b8bdbd5-4trlx" podStartSLOduration=1.3973099900000001 podStartE2EDuration="1.39730999s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:27:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:27:05.394990344 +0000 UTC m=+665.311299772" watchObservedRunningTime="2026-10-31 21:27:05.39730999 +0000 UTC m=+665.313619419"
Oct 31 21:27:05.559569 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:05.559533    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:05.576018 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:05.575995    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-4b5989-7bdb89b88c-qgl9q"]
Oct 31 21:27:06.042892 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.042862    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:06.142031 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:27:06.141993    4460 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8875cc55_d9a9_4759_add2_2ce8d59a4a02.slice/crio-347298440c16e2fbbf4be1c10cddc0eec55f828fc53e8dde89f54d7915f70db0\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75599b5e_0a6e_47e0_8598_39e58ee2edfd.slice/crio-cb7b38bb1aa10032caf4e17bb3604847a2cac10cd76af9838c5d773039aec454.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75599b5e_0a6e_47e0_8598_39e58ee2edfd.slice/crio-conmon-cb7b38bb1aa10032caf4e17bb3604847a2cac10cd76af9838c5d773039aec454.scope\": RecentStats: unable to find data in memory cache]"
Oct 31 21:27:06.186439 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.186401    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:06.244758 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.244696    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:27:06.264401 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.264372    4460 generic.go:356] "Generic (PLEG): container finished" podID="75599b5e-0a6e-47e0-8598-39e58ee2edfd" containerID="cb7b38bb1aa10032caf4e17bb3604847a2cac10cd76af9838c5d773039aec454" exitCode=0
Oct 31 21:27:06.264514 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.264417    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"
Oct 31 21:27:06.264514 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.264455    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2" event={"ID":"75599b5e-0a6e-47e0-8598-39e58ee2edfd","Type":"ContainerDied","Data":"cb7b38bb1aa10032caf4e17bb3604847a2cac10cd76af9838c5d773039aec454"}
Oct 31 21:27:06.264610 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.264516    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2" event={"ID":"75599b5e-0a6e-47e0-8598-39e58ee2edfd","Type":"ContainerDied","Data":"495dfbe25d98ec544cfbcdc2f97d869c8f335348e1f252217f16c4a7e10fbabc"}
Oct 31 21:27:06.264610 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.264532    4460 scope.go:122] "RemoveContainer" containerID="cb7b38bb1aa10032caf4e17bb3604847a2cac10cd76af9838c5d773039aec454"
Oct 31 21:27:06.265999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.265977    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b5989-7bdb89b88c-qgl9q" event={"ID":"4c072735-3c6f-4c6a-9d51-b535fe7f4dfe","Type":"ContainerStarted","Data":"daf570975ab6fb4180d7ddf1c97dc9cb1d678e01954f556c0018426b99defd2f"}
Oct 31 21:27:06.266091 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.266020    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b5989-7bdb89b88c-qgl9q" event={"ID":"4c072735-3c6f-4c6a-9d51-b535fe7f4dfe","Type":"ContainerStarted","Data":"1717acd18afc1aca63cb805b9910f3721aec96da4863bd78f192e25471d00968"}
Oct 31 21:27:06.273405 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.273261    4460 scope.go:122] "RemoveContainer" containerID="8ae36d9c22de57b2c2af20f3e4b4dbe3117d6280358b8470a8fc0a8ce7a30502"
Oct 31 21:27:06.277296 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.277276    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/75599b5e-0a6e-47e0-8598-39e58ee2edfd-kube-api-access-kq4c7\" (UniqueName: \"kubernetes.io/projected/75599b5e-0a6e-47e0-8598-39e58ee2edfd-kube-api-access-kq4c7\") pod \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") "
Oct 31 21:27:06.277391 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.277313    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istio-envoy\") pod \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") "
Oct 31 21:27:06.277391 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.277352    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istio-data\") pod \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") "
Oct 31 21:27:06.277495 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.277411    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istio-token\" (UniqueName: \"kubernetes.io/projected/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istio-token\") pod \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") "
Oct 31 21:27:06.277495 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.277445    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-workload-socket\") pod \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") "
Oct 31 21:27:06.277591 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.277503    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-credential-socket\") pod \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") "
Oct 31 21:27:06.277591 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.277531    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-workload-certs\") pod \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") "
Oct 31 21:27:06.277591 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.277562    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istiod-ca-cert\") pod \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") "
Oct 31 21:27:06.277725 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.277626    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istio-podinfo\") pod \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\" (UID: \"75599b5e-0a6e-47e0-8598-39e58ee2edfd\") "
Oct 31 21:27:06.277725 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.277657    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istio-data" pod "75599b5e-0a6e-47e0-8598-39e58ee2edfd" (UID: "75599b5e-0a6e-47e0-8598-39e58ee2edfd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:27:06.277725 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.277710    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-workload-socket" pod "75599b5e-0a6e-47e0-8598-39e58ee2edfd" (UID: "75599b5e-0a6e-47e0-8598-39e58ee2edfd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:27:06.277879 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.277733    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-credential-socket" pod "75599b5e-0a6e-47e0-8598-39e58ee2edfd" (UID: "75599b5e-0a6e-47e0-8598-39e58ee2edfd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:27:06.277879 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.277859    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-workload-certs" pod "75599b5e-0a6e-47e0-8598-39e58ee2edfd" (UID: "75599b5e-0a6e-47e0-8598-39e58ee2edfd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:27:06.278086 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.278065    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istiod-ca-cert" pod "75599b5e-0a6e-47e0-8598-39e58ee2edfd" (UID: "75599b5e-0a6e-47e0-8598-39e58ee2edfd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:27:06.278386 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.278358    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:27:06.278469 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.278410    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:27:06.278469 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.278425    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:27:06.278469 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.278441    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:27:06.278469 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.278454    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:27:06.279767 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.279745    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75599b5e-0a6e-47e0-8598-39e58ee2edfd-kube-api-access-kq4c7" pod "75599b5e-0a6e-47e0-8598-39e58ee2edfd" (UID: "75599b5e-0a6e-47e0-8598-39e58ee2edfd"). InnerVolumeSpecName "kube-api-access-kq4c7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:27:06.279995 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.279929    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istio-envoy" pod "75599b5e-0a6e-47e0-8598-39e58ee2edfd" (UID: "75599b5e-0a6e-47e0-8598-39e58ee2edfd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:27:06.281403 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.281380    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istio-token" pod "75599b5e-0a6e-47e0-8598-39e58ee2edfd" (UID: "75599b5e-0a6e-47e0-8598-39e58ee2edfd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:27:06.282168 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.282149    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istio-podinfo" pod "75599b5e-0a6e-47e0-8598-39e58ee2edfd" (UID: "75599b5e-0a6e-47e0-8598-39e58ee2edfd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:27:06.282860 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.282845    4460 scope.go:122] "RemoveContainer" containerID="cb7b38bb1aa10032caf4e17bb3604847a2cac10cd76af9838c5d773039aec454"
Oct 31 21:27:06.283179 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:27:06.283160    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb7b38bb1aa10032caf4e17bb3604847a2cac10cd76af9838c5d773039aec454\": container with ID starting with cb7b38bb1aa10032caf4e17bb3604847a2cac10cd76af9838c5d773039aec454 not found: ID does not exist" containerID="cb7b38bb1aa10032caf4e17bb3604847a2cac10cd76af9838c5d773039aec454"
Oct 31 21:27:06.283252 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.283189    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb7b38bb1aa10032caf4e17bb3604847a2cac10cd76af9838c5d773039aec454"} err="failed to get container status \"cb7b38bb1aa10032caf4e17bb3604847a2cac10cd76af9838c5d773039aec454\": rpc error: code = NotFound desc = could not find container \"cb7b38bb1aa10032caf4e17bb3604847a2cac10cd76af9838c5d773039aec454\": container with ID starting with cb7b38bb1aa10032caf4e17bb3604847a2cac10cd76af9838c5d773039aec454 not found: ID does not exist"
Oct 31 21:27:06.283252 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.283211    4460 scope.go:122] "RemoveContainer" containerID="8ae36d9c22de57b2c2af20f3e4b4dbe3117d6280358b8470a8fc0a8ce7a30502"
Oct 31 21:27:06.283487 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:27:06.283468    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ae36d9c22de57b2c2af20f3e4b4dbe3117d6280358b8470a8fc0a8ce7a30502\": container with ID starting with 8ae36d9c22de57b2c2af20f3e4b4dbe3117d6280358b8470a8fc0a8ce7a30502 not found: ID does not exist" containerID="8ae36d9c22de57b2c2af20f3e4b4dbe3117d6280358b8470a8fc0a8ce7a30502"
Oct 31 21:27:06.283531 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.283494    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ae36d9c22de57b2c2af20f3e4b4dbe3117d6280358b8470a8fc0a8ce7a30502"} err="failed to get container status \"8ae36d9c22de57b2c2af20f3e4b4dbe3117d6280358b8470a8fc0a8ce7a30502\": rpc error: code = NotFound desc = could not find container \"8ae36d9c22de57b2c2af20f3e4b4dbe3117d6280358b8470a8fc0a8ce7a30502\": container with ID starting with 8ae36d9c22de57b2c2af20f3e4b4dbe3117d6280358b8470a8fc0a8ce7a30502 not found: ID does not exist"
Oct 31 21:27:06.354769 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.354716    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b5989-7bdb89b88c-qgl9q" podStartSLOduration=2.354700442 podStartE2EDuration="2.354700442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:27:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:27:06.352746571 +0000 UTC m=+666.269056003" watchObservedRunningTime="2026-10-31 21:27:06.354700442 +0000 UTC m=+666.271009871"
Oct 31 21:27:06.380103 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.380071    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:27:06.380103 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.380104    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kq4c7\" (UniqueName: \"kubernetes.io/projected/75599b5e-0a6e-47e0-8598-39e58ee2edfd-kube-api-access-kq4c7\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:27:06.380319 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.380119    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:27:06.380319 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.380133    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75599b5e-0a6e-47e0-8598-39e58ee2edfd-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:27:06.414380 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.414347    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:06.549014 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.548971    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:06.613542 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.613513    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"]
Oct 31 21:27:06.634004 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:06.630734    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-ac0736b6-openshift-default-65bc984bbf-vrcz2"]
Oct 31 21:27:07.236547 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:07.236485    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:07.273776 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:07.273748    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b5989-7bdb89b88c-qgl9q"
Oct 31 21:27:07.431739 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:07.431706    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:07.558571 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:07.558533    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:07.697372 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:07.697336    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:07.820953 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:07.813214    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:08.012758 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:08.012724    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:08.224029 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:08.223931    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="75599b5e-0a6e-47e0-8598-39e58ee2edfd" path="/var/lib/kubelet/pods/75599b5e-0a6e-47e0-8598-39e58ee2edfd/volumes"
Oct 31 21:27:08.876581 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:08.876548    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:08.993419 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:08.993373    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:09.044997 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:09.043484    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-c16684-94dc7cbd7-gjj44"]
Oct 31 21:27:09.044997 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:09.044130    4460 cpu_manager.go:414] "containerMap: removing container" podUID="75599b5e-0a6e-47e0-8598-39e58ee2edfd" containerName="istio-proxy"
Oct 31 21:27:09.044997 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:09.044147    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="75599b5e-0a6e-47e0-8598-39e58ee2edfd" containerName="istio-proxy"
Oct 31 21:27:09.044997 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:09.044175    4460 cpu_manager.go:414] "containerMap: removing container" podUID="75599b5e-0a6e-47e0-8598-39e58ee2edfd" containerName="istio-proxy"
Oct 31 21:27:09.044997 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:09.044192    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="75599b5e-0a6e-47e0-8598-39e58ee2edfd" containerName="istio-proxy"
Oct 31 21:27:09.044997 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:09.044313    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="75599b5e-0a6e-47e0-8598-39e58ee2edfd" containerName="istio-proxy"
Oct 31 21:27:09.044997 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:09.044330    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="75599b5e-0a6e-47e0-8598-39e58ee2edfd" containerName="istio-proxy"
Oct 31 21:27:09.049404 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:09.049381    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c16684-94dc7cbd7-gjj44"
Oct 31 21:27:09.065721 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:09.065698    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-c16684-94dc7cbd7-gjj44"]
Oct 31 21:27:09.077326 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:09.077309    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w2-c16684-dockercfg-m88z9\""
Oct 31 21:27:09.077326 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:09.077314    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w2-c16684\""
Oct 31 21:27:09.107104 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:09.107075    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-c16684-685b865b6c-qbtqr"]
Oct 31 21:27:09.107484 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:09.107459    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/076218ca-4ba5-43dc-967e-fb30eda6e029-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-c16684-94dc7cbd7-gjj44\" (UID: \"076218ca-4ba5-43dc-967e-fb30eda6e029\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c16684-94dc7cbd7-gjj44"
Oct 31 21:27:09.107558 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:09.107512    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmdgz\" (UniqueName: \"kubernetes.io/projected/076218ca-4ba5-43dc-967e-fb30eda6e029-kube-api-access-lmdgz\") pod \"payload-pre-processing-e2e-worker-w2-c16684-94dc7cbd7-gjj44\" (UID: \"076218ca-4ba5-43dc-967e-fb30eda6e029\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c16684-94dc7cbd7-gjj44"
Oct 31 21:27:09.111379 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:09.111359    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-c16684-685b865b6c-qbtqr"
Oct 31 21:27:09.137960 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:09.137912    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-c16684-685b865b6c-qbtqr"]
Oct 31 21:27:09.208411 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:09.208227    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/076218ca-4ba5-43dc-967e-fb30eda6e029-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-c16684-94dc7cbd7-gjj44\" (UID: \"076218ca-4ba5-43dc-967e-fb30eda6e029\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c16684-94dc7cbd7-gjj44"
Oct 31 21:27:09.208411 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:09.208287    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2chtv\" (UniqueName: \"kubernetes.io/projected/5ce9718e-7d7b-4fc4-b506-b25cd880efc3-kube-api-access-2chtv\") pod \"payload-processing-e2e-worker-w2-c16684-685b865b6c-qbtqr\" (UID: \"5ce9718e-7d7b-4fc4-b506-b25cd880efc3\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-c16684-685b865b6c-qbtqr"
Oct 31 21:27:09.208411 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:09.208336    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmdgz\" (UniqueName: \"kubernetes.io/projected/076218ca-4ba5-43dc-967e-fb30eda6e029-kube-api-access-lmdgz\") pod \"payload-pre-processing-e2e-worker-w2-c16684-94dc7cbd7-gjj44\" (UID: \"076218ca-4ba5-43dc-967e-fb30eda6e029\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c16684-94dc7cbd7-gjj44"
Oct 31 21:27:09.208411 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:09.208395    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ce9718e-7d7b-4fc4-b506-b25cd880efc3-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-c16684-685b865b6c-qbtqr\" (UID: \"5ce9718e-7d7b-4fc4-b506-b25cd880efc3\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-c16684-685b865b6c-qbtqr"
Oct 31 21:27:09.209793 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:09.209756    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/076218ca-4ba5-43dc-967e-fb30eda6e029-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-c16684-94dc7cbd7-gjj44\" (UID: \"076218ca-4ba5-43dc-967e-fb30eda6e029\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c16684-94dc7cbd7-gjj44"
Oct 31 21:27:09.247067 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:09.247031    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmdgz\" (UniqueName: \"kubernetes.io/projected/076218ca-4ba5-43dc-967e-fb30eda6e029-kube-api-access-lmdgz\") pod \"payload-pre-processing-e2e-worker-w2-c16684-94dc7cbd7-gjj44\" (UID: \"076218ca-4ba5-43dc-967e-fb30eda6e029\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c16684-94dc7cbd7-gjj44"
Oct 31 21:27:09.309551 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:09.309517    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ce9718e-7d7b-4fc4-b506-b25cd880efc3-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-c16684-685b865b6c-qbtqr\" (UID: \"5ce9718e-7d7b-4fc4-b506-b25cd880efc3\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-c16684-685b865b6c-qbtqr"
Oct 31 21:27:09.309730 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:09.309680    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2chtv\" (UniqueName: \"kubernetes.io/projected/5ce9718e-7d7b-4fc4-b506-b25cd880efc3-kube-api-access-2chtv\") pod \"payload-processing-e2e-worker-w2-c16684-685b865b6c-qbtqr\" (UID: \"5ce9718e-7d7b-4fc4-b506-b25cd880efc3\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-c16684-685b865b6c-qbtqr"
Oct 31 21:27:09.310347 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:09.310324    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ce9718e-7d7b-4fc4-b506-b25cd880efc3-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-c16684-685b865b6c-qbtqr\" (UID: \"5ce9718e-7d7b-4fc4-b506-b25cd880efc3\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-c16684-685b865b6c-qbtqr"
Oct 31 21:27:09.342962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:09.341784    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2chtv\" (UniqueName: \"kubernetes.io/projected/5ce9718e-7d7b-4fc4-b506-b25cd880efc3-kube-api-access-2chtv\") pod \"payload-processing-e2e-worker-w2-c16684-685b865b6c-qbtqr\" (UID: \"5ce9718e-7d7b-4fc4-b506-b25cd880efc3\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-c16684-685b865b6c-qbtqr"
Oct 31 21:27:09.366716 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:09.365171    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c16684-94dc7cbd7-gjj44"
Oct 31 21:27:09.507362 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:09.507328    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-c16684-685b865b6c-qbtqr"
Oct 31 21:27:09.577456 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:09.577396    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:09.776129 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:09.775980    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-c16684-94dc7cbd7-gjj44"]
Oct 31 21:27:09.779685 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:27:09.779647    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod076218ca_4ba5_43dc_967e_fb30eda6e029.slice/crio-897f3d98fad18551cb35ec5d998a6d54391aa254695f9536b8ab01362b118e96 WatchSource:0}: Error finding container 897f3d98fad18551cb35ec5d998a6d54391aa254695f9536b8ab01362b118e96: Status 404 returned error can't find the container with id 897f3d98fad18551cb35ec5d998a6d54391aa254695f9536b8ab01362b118e96
Oct 31 21:27:09.943260 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:09.943168    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-c16684-685b865b6c-qbtqr"]
Oct 31 21:27:09.974398 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:27:09.974345    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ce9718e_7d7b_4fc4_b506_b25cd880efc3.slice/crio-89645908b2bc753769f43ad47aec0845ddd75c6ea50e9aa93278aab9c034972c WatchSource:0}: Error finding container 89645908b2bc753769f43ad47aec0845ddd75c6ea50e9aa93278aab9c034972c: Status 404 returned error can't find the container with id 89645908b2bc753769f43ad47aec0845ddd75c6ea50e9aa93278aab9c034972c
Oct 31 21:27:10.298762 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:10.298664    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-c16684-685b865b6c-qbtqr" event={"ID":"5ce9718e-7d7b-4fc4-b506-b25cd880efc3","Type":"ContainerStarted","Data":"a64a5862408480fe92d984ec50e36cbae130e0ad181b98ac1ba7b388f3210fa1"}
Oct 31 21:27:10.298762 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:10.298728    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-c16684-685b865b6c-qbtqr" event={"ID":"5ce9718e-7d7b-4fc4-b506-b25cd880efc3","Type":"ContainerStarted","Data":"89645908b2bc753769f43ad47aec0845ddd75c6ea50e9aa93278aab9c034972c"}
Oct 31 21:27:10.299016 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:10.298779    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-c16684-685b865b6c-qbtqr"
Oct 31 21:27:10.300198 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:10.300177    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c16684-94dc7cbd7-gjj44" event={"ID":"076218ca-4ba5-43dc-967e-fb30eda6e029","Type":"ContainerStarted","Data":"a73559c973889aff2df223ab52e35a5f92a5f300f4069a71761fa1ff23333c6d"}
Oct 31 21:27:10.300248 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:10.300217    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c16684-94dc7cbd7-gjj44" event={"ID":"076218ca-4ba5-43dc-967e-fb30eda6e029","Type":"ContainerStarted","Data":"897f3d98fad18551cb35ec5d998a6d54391aa254695f9536b8ab01362b118e96"}
Oct 31 21:27:10.300657 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:10.300634    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c16684-94dc7cbd7-gjj44"
Oct 31 21:27:10.372992 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:10.370496    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w2-c16684-685b865b6c-qbtqr" podStartSLOduration=1.370480439 podStartE2EDuration="1.370480439s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:27:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:27:10.346349915 +0000 UTC m=+670.262659345" watchObservedRunningTime="2026-10-31 21:27:10.370480439 +0000 UTC m=+670.286789867"
Oct 31 21:27:10.403166 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:10.403091    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c16684-94dc7cbd7-gjj44" podStartSLOduration=1.403071524 podStartE2EDuration="1.403071524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:27:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:27:10.400427827 +0000 UTC m=+670.316737256" watchObservedRunningTime="2026-10-31 21:27:10.403071524 +0000 UTC m=+670.319380954"
Oct 31 21:27:10.660018 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:27:10.659373    4460 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8875cc55_d9a9_4759_add2_2ce8d59a4a02.slice/crio-347298440c16e2fbbf4be1c10cddc0eec55f828fc53e8dde89f54d7915f70db0\": RecentStats: unable to find data in memory cache]"
Oct 31 21:27:10.815167 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:10.815133    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:10.971294 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:10.971073    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:11.013979 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:11.013931    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-cc45a7-86f7d9c8c9-lpr6x"]
Oct 31 21:27:11.017892 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:11.017870    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cc45a7-86f7d9c8c9-lpr6x"
Oct 31 21:27:11.060966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:11.052678    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b6a669d-e3cb-405b-a1c4-6fdfceb78990-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-cc45a7-86f7d9c8c9-lpr6x\" (UID: \"2b6a669d-e3cb-405b-a1c4-6fdfceb78990\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cc45a7-86f7d9c8c9-lpr6x"
Oct 31 21:27:11.060966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:11.052779    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvxz7\" (UniqueName: \"kubernetes.io/projected/2b6a669d-e3cb-405b-a1c4-6fdfceb78990-kube-api-access-mvxz7\") pod \"payload-pre-processing-e2e-worker-w4-cc45a7-86f7d9c8c9-lpr6x\" (UID: \"2b6a669d-e3cb-405b-a1c4-6fdfceb78990\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cc45a7-86f7d9c8c9-lpr6x"
Oct 31 21:27:11.074962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:11.062176    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-cc45a7-dockercfg-6qh7z\""
Oct 31 21:27:11.074962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:11.062277    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-cc45a7\""
Oct 31 21:27:11.074962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:11.066691    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-cc45a7-86f7d9c8c9-lpr6x"]
Oct 31 21:27:11.177341 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:11.155648    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b6a669d-e3cb-405b-a1c4-6fdfceb78990-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-cc45a7-86f7d9c8c9-lpr6x\" (UID: \"2b6a669d-e3cb-405b-a1c4-6fdfceb78990\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cc45a7-86f7d9c8c9-lpr6x"
Oct 31 21:27:11.177341 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:11.155743    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvxz7\" (UniqueName: \"kubernetes.io/projected/2b6a669d-e3cb-405b-a1c4-6fdfceb78990-kube-api-access-mvxz7\") pod \"payload-pre-processing-e2e-worker-w4-cc45a7-86f7d9c8c9-lpr6x\" (UID: \"2b6a669d-e3cb-405b-a1c4-6fdfceb78990\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cc45a7-86f7d9c8c9-lpr6x"
Oct 31 21:27:11.177341 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:11.156982    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b6a669d-e3cb-405b-a1c4-6fdfceb78990-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-cc45a7-86f7d9c8c9-lpr6x\" (UID: \"2b6a669d-e3cb-405b-a1c4-6fdfceb78990\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cc45a7-86f7d9c8c9-lpr6x"
Oct 31 21:27:11.245006 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:11.244897    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvxz7\" (UniqueName: \"kubernetes.io/projected/2b6a669d-e3cb-405b-a1c4-6fdfceb78990-kube-api-access-mvxz7\") pod \"payload-pre-processing-e2e-worker-w4-cc45a7-86f7d9c8c9-lpr6x\" (UID: \"2b6a669d-e3cb-405b-a1c4-6fdfceb78990\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cc45a7-86f7d9c8c9-lpr6x"
Oct 31 21:27:11.284448 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:11.284412    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn"
Oct 31 21:27:11.381888 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:11.381854    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cc45a7-86f7d9c8c9-lpr6x"
Oct 31 21:27:11.582199 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:11.582156    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:11.704648 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:11.704612    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:11.806037 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:11.805904    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-cc45a7-86f7d9c8c9-lpr6x"]
Oct 31 21:27:11.806788 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:27:11.806593    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b6a669d_e3cb_405b_a1c4_6fdfceb78990.slice/crio-3e093649f4f3e93b624f8d8e555218ac1d60cdefbc4b3334eed89aabfd368a06 WatchSource:0}: Error finding container 3e093649f4f3e93b624f8d8e555218ac1d60cdefbc4b3334eed89aabfd368a06: Status 404 returned error can't find the container with id 3e093649f4f3e93b624f8d8e555218ac1d60cdefbc4b3334eed89aabfd368a06
Oct 31 21:27:11.905964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:11.890699    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:11.905964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:11.900637    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:27:12.073016 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:12.059178    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:12.311829 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:12.311789    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cc45a7-86f7d9c8c9-lpr6x" event={"ID":"2b6a669d-e3cb-405b-a1c4-6fdfceb78990","Type":"ContainerStarted","Data":"57633fd1ff45c42392db0018098b0cf779ab0b90dd4c9fb64f88e3a8e6086d78"}
Oct 31 21:27:12.312052 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:12.311847    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cc45a7-86f7d9c8c9-lpr6x" event={"ID":"2b6a669d-e3cb-405b-a1c4-6fdfceb78990","Type":"ContainerStarted","Data":"3e093649f4f3e93b624f8d8e555218ac1d60cdefbc4b3334eed89aabfd368a06"}
Oct 31 21:27:13.317325 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:13.317253    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cc45a7-86f7d9c8c9-lpr6x"
Oct 31 21:27:13.555763 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:13.555729    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:14.803905 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:14.803759    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:14.927229 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:14.927189    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:15.281127 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:15.281072    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cc45a7-86f7d9c8c9-lpr6x" podStartSLOduration=5.28105607 podStartE2EDuration="5.28105607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:27:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:27:12.37019369 +0000 UTC m=+672.286503119" watchObservedRunningTime="2026-10-31 21:27:15.28105607 +0000 UTC m=+675.197365498"
Oct 31 21:27:15.281779 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:15.281755    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc"]
Oct 31 21:27:15.285638 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:15.285615    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc"
Oct 31 21:27:15.305431 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:15.305408    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-cc45a7-serving-cert\""
Oct 31 21:27:15.321554 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:15.321523    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc"]
Oct 31 21:27:15.383135 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:15.383064    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b369e84-9944-4d3b-9bc1-9aa885694d11-metrics-certs\") pod \"maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc\" (UID: \"8b369e84-9944-4d3b-9bc1-9aa885694d11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc"
Oct 31 21:27:15.383135 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:15.383120    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b369e84-9944-4d3b-9bc1-9aa885694d11-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc\" (UID: \"8b369e84-9944-4d3b-9bc1-9aa885694d11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc"
Oct 31 21:27:15.383402 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:15.383193    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b369e84-9944-4d3b-9bc1-9aa885694d11-maas-api-tls\") pod \"maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc\" (UID: \"8b369e84-9944-4d3b-9bc1-9aa885694d11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc"
Oct 31 21:27:15.383402 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:15.383327    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mf7f\" (UniqueName: \"kubernetes.io/projected/8b369e84-9944-4d3b-9bc1-9aa885694d11-kube-api-access-8mf7f\") pod \"maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc\" (UID: \"8b369e84-9944-4d3b-9bc1-9aa885694d11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc"
Oct 31 21:27:15.401183 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:15.401152    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:15.486628 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:15.484183    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b369e84-9944-4d3b-9bc1-9aa885694d11-maas-api-tls\") pod \"maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc\" (UID: \"8b369e84-9944-4d3b-9bc1-9aa885694d11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc"
Oct 31 21:27:15.486628 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:15.484280    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mf7f\" (UniqueName: \"kubernetes.io/projected/8b369e84-9944-4d3b-9bc1-9aa885694d11-kube-api-access-8mf7f\") pod \"maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc\" (UID: \"8b369e84-9944-4d3b-9bc1-9aa885694d11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc"
Oct 31 21:27:15.486628 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:15.484399    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b369e84-9944-4d3b-9bc1-9aa885694d11-metrics-certs\") pod \"maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc\" (UID: \"8b369e84-9944-4d3b-9bc1-9aa885694d11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc"
Oct 31 21:27:15.486628 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:15.484429    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b369e84-9944-4d3b-9bc1-9aa885694d11-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc\" (UID: \"8b369e84-9944-4d3b-9bc1-9aa885694d11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc"
Oct 31 21:27:15.486628 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:15.485273    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b369e84-9944-4d3b-9bc1-9aa885694d11-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc\" (UID: \"8b369e84-9944-4d3b-9bc1-9aa885694d11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc"
Oct 31 21:27:15.486628 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:27:15.485647    4460 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-cc45a7-serving-cert: secret "maas-api-e2e-worker-w4-cc45a7-serving-cert" not found
Oct 31 21:27:15.486628 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:27:15.485713    4460 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b369e84-9944-4d3b-9bc1-9aa885694d11-maas-api-tls podName:8b369e84-9944-4d3b-9bc1-9aa885694d11 nodeName:}" failed. No retries permitted until 2026-10-31 21:27:15.985690253 +0000 UTC m=+675.901999676 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8b369e84-9944-4d3b-9bc1-9aa885694d11-maas-api-tls") pod "maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc" (UID: "8b369e84-9944-4d3b-9bc1-9aa885694d11") : secret "maas-api-e2e-worker-w4-cc45a7-serving-cert" not found
Oct 31 21:27:15.507124 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:15.507092    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b369e84-9944-4d3b-9bc1-9aa885694d11-metrics-certs\") pod \"maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc\" (UID: \"8b369e84-9944-4d3b-9bc1-9aa885694d11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc"
Oct 31 21:27:15.523024 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:15.522958    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mf7f\" (UniqueName: \"kubernetes.io/projected/8b369e84-9944-4d3b-9bc1-9aa885694d11-kube-api-access-8mf7f\") pod \"maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc\" (UID: \"8b369e84-9944-4d3b-9bc1-9aa885694d11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc"
Oct 31 21:27:15.542388 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:15.542244    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:15.662826 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:15.662774    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:15.989727 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:15.989647    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b369e84-9944-4d3b-9bc1-9aa885694d11-maas-api-tls\") pod \"maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc\" (UID: \"8b369e84-9944-4d3b-9bc1-9aa885694d11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc"
Oct 31 21:27:15.992262 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:15.992217    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b369e84-9944-4d3b-9bc1-9aa885694d11-maas-api-tls\") pod \"maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc\" (UID: \"8b369e84-9944-4d3b-9bc1-9aa885694d11\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc"
Oct 31 21:27:16.199981 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:16.199923    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc"
Oct 31 21:27:16.217846 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:27:16.217812    4460 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8875cc55_d9a9_4759_add2_2ce8d59a4a02.slice/crio-347298440c16e2fbbf4be1c10cddc0eec55f828fc53e8dde89f54d7915f70db0\": RecentStats: unable to find data in memory cache]"
Oct 31 21:27:16.267334 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:16.257157    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 21:27:16.267334 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:16.257223    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9" podUID="9501cf33-637a-4d00-8e58-3ae157dd6840" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:27:16.288716 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:16.283404    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-106555-659c88c8f5-7hdjm"
Oct 31 21:27:16.290585 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:16.289550    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-106555-849b8bdbd5-4trlx"
Oct 31 21:27:16.521056 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:16.521021    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc"]
Oct 31 21:27:16.522719 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:27:16.522676    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b369e84_9944_4d3b_9bc1_9aa885694d11.slice/crio-98bdeb2c7c523f9e305ebc83315ea30c126de9939102879f77242f76c59f844d WatchSource:0}: Error finding container 98bdeb2c7c523f9e305ebc83315ea30c126de9939102879f77242f76c59f844d: Status 404 returned error can't find the container with id 98bdeb2c7c523f9e305ebc83315ea30c126de9939102879f77242f76c59f844d
Oct 31 21:27:16.587825 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:16.587786    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:17.171282 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:17.171186    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:17.294186 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:17.294151    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:17.374659 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:17.374621    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc" event={"ID":"8b369e84-9944-4d3b-9bc1-9aa885694d11","Type":"ContainerStarted","Data":"d68596f112e9b44ed304cba2d00e33f292dbf9ae8db65672ece3675581e2f074"}
Oct 31 21:27:17.374856 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:17.374675    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc" event={"ID":"8b369e84-9944-4d3b-9bc1-9aa885694d11","Type":"ContainerStarted","Data":"98bdeb2c7c523f9e305ebc83315ea30c126de9939102879f77242f76c59f844d"}
Oct 31 21:27:17.374856 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:17.374707    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc"
Oct 31 21:27:17.424043 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:17.423978    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:17.425157 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:17.424595    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc" podStartSLOduration=2.065510589 podStartE2EDuration="2.424582938s" totalImagesPullingTime="359.072349ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:27:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:27:17.421671767 +0000 UTC m=+677.337981196" watchObservedRunningTime="2026-10-31 21:27:17.424582938 +0000 UTC m=+677.340892401"
Oct 31 21:27:18.288635 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:18.288604    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b5989-7bdb89b88c-qgl9q"
Oct 31 21:27:18.382115 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:18.382085    4460 generic.go:356] "Generic (PLEG): container finished" podID="ea04c855-12eb-4628-a947-3b4e8321bef1" containerID="9cd7432e1359bfff5b62fbe8a1c9facffb89c649f1993ea889ced7ada81fd433" exitCode=0
Oct 31 21:27:18.382311 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:18.382159    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z" event={"ID":"ea04c855-12eb-4628-a947-3b4e8321bef1","Type":"ContainerDied","Data":"9cd7432e1359bfff5b62fbe8a1c9facffb89c649f1993ea889ced7ada81fd433"}
Oct 31 21:27:18.382603 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:18.382586    4460 scope.go:122] "RemoveContainer" containerID="9cd7432e1359bfff5b62fbe8a1c9facffb89c649f1993ea889ced7ada81fd433"
Oct 31 21:27:18.384087 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:18.384054    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:27:18.384189 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:18.384142    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:27:18.384229 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:18.384186    4460 generic.go:356] "Generic (PLEG): container finished" podID="9501cf33-637a-4d00-8e58-3ae157dd6840" containerID="e2c939512586f906bab898a18a9dabd22b7a1e5615e0e076315b70cc71573229" exitCode=0
Oct 31 21:27:18.384229 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:18.384212    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:27:18.384291 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:18.384258    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9" event={"ID":"9501cf33-637a-4d00-8e58-3ae157dd6840","Type":"ContainerDied","Data":"e2c939512586f906bab898a18a9dabd22b7a1e5615e0e076315b70cc71573229"}
Oct 31 21:27:18.384619 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:18.384603    4460 scope.go:122] "RemoveContainer" containerID="e2c939512586f906bab898a18a9dabd22b7a1e5615e0e076315b70cc71573229"
Oct 31 21:27:18.385699 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:18.385674    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:27:18.385792 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:18.385745    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:27:18.385853 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:18.385791    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:27:19.305192 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:19.305155    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:19.395385 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:19.395347    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z" event={"ID":"ea04c855-12eb-4628-a947-3b4e8321bef1","Type":"ContainerStarted","Data":"b816bf29bf7a227b1ab47d8eae86b3be3f257d6564a18b8afc3556945da1757f"}
Oct 31 21:27:19.397134 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:19.397112    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9" event={"ID":"9501cf33-637a-4d00-8e58-3ae157dd6840","Type":"ContainerStarted","Data":"ff0ac493ac8a0d8677ef67fc5cc9d3069b738c9ef3338f07eb651fb194cc4a8a"}
Oct 31 21:27:19.558024 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:19.557884    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:20.158963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:20.158338    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:27:20.158963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:20.158473    4460 patch_prober.go:29] interesting pod/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Oct 31 21:27:20.158963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:20.158524    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z" podUID="ea04c855-12eb-4628-a947-3b4e8321bef1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Oct 31 21:27:20.261326 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:20.261293    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:27:20.261662 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:20.261637    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Oct 31 21:27:20.261837 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:20.261811    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9" podUID="9501cf33-637a-4d00-8e58-3ae157dd6840" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Oct 31 21:27:20.930879 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:20.921751    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx"]
Oct 31 21:27:20.945047 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:20.945020    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx"
Oct 31 21:27:20.968581 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:20.968557    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-994336-serving-cert\""
Oct 31 21:27:20.985658 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:20.982176    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx"]
Oct 31 21:27:20.994673 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:20.994645    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j98zs\" (UniqueName: \"kubernetes.io/projected/aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac-kube-api-access-j98zs\") pod \"maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx\" (UID: \"aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx"
Oct 31 21:27:20.994814 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:20.994684    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx\" (UID: \"aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx"
Oct 31 21:27:20.994814 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:20.994711    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac-maas-api-tls\") pod \"maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx\" (UID: \"aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx"
Oct 31 21:27:20.994934 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:20.994854    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac-metrics-certs\") pod \"maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx\" (UID: \"aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx"
Oct 31 21:27:21.046691 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:21.046660    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:21.095915 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:21.095873    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac-metrics-certs\") pod \"maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx\" (UID: \"aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx"
Oct 31 21:27:21.096098 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:21.095968    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j98zs\" (UniqueName: \"kubernetes.io/projected/aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac-kube-api-access-j98zs\") pod \"maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx\" (UID: \"aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx"
Oct 31 21:27:21.096314 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:21.096292    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx\" (UID: \"aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx"
Oct 31 21:27:21.096366 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:21.096349    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac-maas-api-tls\") pod \"maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx\" (UID: \"aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx"
Oct 31 21:27:21.096533 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:27:21.096519    4460 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w1-994336-serving-cert: secret "maas-api-e2e-worker-w1-994336-serving-cert" not found
Oct 31 21:27:21.096575 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:27:21.096570    4460 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac-maas-api-tls podName:aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac nodeName:}" failed. No retries permitted until 2026-10-31 21:27:21.596556003 +0000 UTC m=+681.512865411 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac-maas-api-tls") pod "maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx" (UID: "aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac") : secret "maas-api-e2e-worker-w1-994336-serving-cert" not found
Oct 31 21:27:21.097167 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:21.097145    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx\" (UID: \"aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx"
Oct 31 21:27:21.098967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:21.098647    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac-metrics-certs\") pod \"maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx\" (UID: \"aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx"
Oct 31 21:27:21.132961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:21.131693    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j98zs\" (UniqueName: \"kubernetes.io/projected/aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac-kube-api-access-j98zs\") pod \"maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx\" (UID: \"aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx"
Oct 31 21:27:21.155964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:21.155670    4460 patch_prober.go:29] interesting pod/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Oct 31 21:27:21.155964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:21.155724    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z" podUID="ea04c855-12eb-4628-a947-3b4e8321bef1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Oct 31 21:27:21.163960 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:21.163365    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:21.256020 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:21.255415    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Oct 31 21:27:21.256020 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:21.255481    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9" podUID="9501cf33-637a-4d00-8e58-3ae157dd6840" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Oct 31 21:27:21.287557 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:21.287282    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:21.337963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:21.325698    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-c16684-685b865b6c-qbtqr"
Oct 31 21:27:21.502309 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:27:21.501999    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:27:21.603332 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:21.603298    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac-maas-api-tls\") pod \"maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx\" (UID: \"aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx"
Oct 31 21:27:21.605639 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:21.605616    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac-maas-api-tls\") pod \"maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx\" (UID: \"aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx"
Oct 31 21:27:21.786958 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:21.786898    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:27:21.887421 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:21.887333    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx"
Oct 31 21:27:22.154017 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:22.153467    4460 patch_prober.go:29] interesting pod/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Oct 31 21:27:22.154017 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:22.153528    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z" podUID="ea04c855-12eb-4628-a947-3b4e8321bef1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Oct 31 21:27:22.263863 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:22.258375    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Oct 31 21:27:22.263863 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:22.258433    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9" podUID="9501cf33-637a-4d00-8e58-3ae157dd6840" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Oct 31 21:27:22.268488 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:22.268459    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx"]
Oct 31 21:27:22.278463 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:27:22.278422    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaefc68e6_fb60_4fc3_b791_1c6e0a6bf4ac.slice/crio-a2b9a43a12b47e970f4457570ac1ec5353857db33341cf9d0e6923fe0050f44c WatchSource:0}: Error finding container a2b9a43a12b47e970f4457570ac1ec5353857db33341cf9d0e6923fe0050f44c: Status 404 returned error can't find the container with id a2b9a43a12b47e970f4457570ac1ec5353857db33341cf9d0e6923fe0050f44c
Oct 31 21:27:22.313934 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:22.313910    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c16684-94dc7cbd7-gjj44"
Oct 31 21:27:22.483065 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:22.482978    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx" event={"ID":"aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac","Type":"ContainerStarted","Data":"a2b9a43a12b47e970f4457570ac1ec5353857db33341cf9d0e6923fe0050f44c"}
Oct 31 21:27:23.159022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:23.158033    4460 patch_prober.go:29] interesting pod/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Oct 31 21:27:23.159022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:23.158093    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z" podUID="ea04c855-12eb-4628-a947-3b4e8321bef1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Oct 31 21:27:23.260193 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:23.244926    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:23.260193 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:23.257373    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Oct 31 21:27:23.260193 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:23.257437    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9" podUID="9501cf33-637a-4d00-8e58-3ae157dd6840" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Oct 31 21:27:23.390985 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:23.390385    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc"
Oct 31 21:27:23.495040 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:23.492961    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx" event={"ID":"aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac","Type":"ContainerStarted","Data":"6a1eadbfa4128a99de2c6750cf4a66f6626a55ce57d31b20cbe56d993ef7e1f5"}
Oct 31 21:27:23.544962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:23.544326    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx" podStartSLOduration=3.14807235 podStartE2EDuration="3.544304826s" totalImagesPullingTime="396.232476ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:27:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:27:23.543325856 +0000 UTC m=+683.459635285" watchObservedRunningTime="2026-10-31 21:27:23.544304826 +0000 UTC m=+683.460614261"
Oct 31 21:27:24.171964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:24.170076    4460 patch_prober.go:29] interesting pod/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Oct 31 21:27:24.171964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:24.170134    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z" podUID="ea04c855-12eb-4628-a947-3b4e8321bef1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Oct 31 21:27:24.258370 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:24.258334    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Oct 31 21:27:24.258661 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:24.258630    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9" podUID="9501cf33-637a-4d00-8e58-3ae157dd6840" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Oct 31 21:27:24.356643 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:24.356612    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cc45a7-86f7d9c8c9-lpr6x"
Oct 31 21:27:24.500760 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:24.500680    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx"
Oct 31 21:27:25.157927 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:25.157897    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:27:25.158194 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:25.158172    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:27:25.158845 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:25.158824    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 21:27:25.166720 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:25.166696    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:25.259569 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:25.259541    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:27:25.259958 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:25.259825    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:27:25.260573 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:25.260559    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:27:25.702112 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:27:25.702084    4460 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8875cc55_d9a9_4759_add2_2ce8d59a4a02.slice/crio-347298440c16e2fbbf4be1c10cddc0eec55f828fc53e8dde89f54d7915f70db0\": RecentStats: unable to find data in memory cache]"
Oct 31 21:27:25.829712 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:25.829681    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:26.231557 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:27:26.231519    4460 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8875cc55_d9a9_4759_add2_2ce8d59a4a02.slice/crio-347298440c16e2fbbf4be1c10cddc0eec55f828fc53e8dde89f54d7915f70db0\": RecentStats: unable to find data in memory cache]"
Oct 31 21:27:26.351807 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:26.351766    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:26.479486 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:26.479451    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:26.622018 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:26.621983    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:26.977707 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:26.977614    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:27.104441 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:27.104403    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:27.232576 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:27.232493    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:28.974612 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:28.974578    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:30.165447 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:30.165406    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:30.511962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:30.511919    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx"
Oct 31 21:27:31.064791 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:31.064750    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:32.482113 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:32.482082    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:33.002260 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:33.002224    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:33.114135 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:33.114105    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:33.280884 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:33.280843    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:33.455758 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:33.455726    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:27:35.032983 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:35.032931    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:36.286444 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:27:36.286407    4460 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8875cc55_d9a9_4759_add2_2ce8d59a4a02.slice/crio-347298440c16e2fbbf4be1c10cddc0eec55f828fc53e8dde89f54d7915f70db0\": RecentStats: unable to find data in memory cache]"
Oct 31 21:27:36.586827 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:36.586727    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:36.828964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:36.828901    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:37.186315 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:37.186263    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:37.316657 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:37.316625    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:37.438165 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:37.438072    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:38.118288 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:38.118251    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:38.748209 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:38.748167    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:40.451473 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:40.451438    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:40.652410 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:27:40.652374    4460 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8875cc55_d9a9_4759_add2_2ce8d59a4a02.slice/crio-347298440c16e2fbbf4be1c10cddc0eec55f828fc53e8dde89f54d7915f70db0\": RecentStats: unable to find data in memory cache]"
Oct 31 21:27:41.196222 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:27:41.196186    4460 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8875cc55_d9a9_4759_add2_2ce8d59a4a02.slice/crio-347298440c16e2fbbf4be1c10cddc0eec55f828fc53e8dde89f54d7915f70db0\": RecentStats: unable to find data in memory cache]"
Oct 31 21:27:41.196422 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:27:41.196280    4460 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8875cc55_d9a9_4759_add2_2ce8d59a4a02.slice/crio-347298440c16e2fbbf4be1c10cddc0eec55f828fc53e8dde89f54d7915f70db0\": RecentStats: unable to find data in memory cache]"
Oct 31 21:27:42.041611 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:42.041569    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:42.358067 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:42.357981    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:44.036360 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:44.036329    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:44.291303 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:44.291226    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:45.714596 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:45.714560    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:46.340584 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:27:46.340554    4460 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8875cc55_d9a9_4759_add2_2ce8d59a4a02.slice/crio-347298440c16e2fbbf4be1c10cddc0eec55f828fc53e8dde89f54d7915f70db0\": RecentStats: unable to find data in memory cache]"
Oct 31 21:27:46.953838 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:46.953803    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:27:47.434761 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:47.434714    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:48.518804 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:48.518764    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:49.217743 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:49.217710    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:49.332389 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:49.332351    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:50.764472 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:50.764422    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:50.878365 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:50.878333    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:50.995429 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:50.995396    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:51.451959 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:27:51.451912    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:27:52.733839 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:52.733805    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:52.949676 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:52.949634    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:54.431717 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:54.431675    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:55.700861 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:27:55.700829    4460 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8875cc55_d9a9_4759_add2_2ce8d59a4a02.slice/crio-347298440c16e2fbbf4be1c10cddc0eec55f828fc53e8dde89f54d7915f70db0\": RecentStats: unable to find data in memory cache]"
Oct 31 21:27:56.032896 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:56.032866    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:56.351448 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:27:56.351365    4460 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8875cc55_d9a9_4759_add2_2ce8d59a4a02.slice/crio-347298440c16e2fbbf4be1c10cddc0eec55f828fc53e8dde89f54d7915f70db0\": RecentStats: unable to find data in memory cache]"
Oct 31 21:27:57.710592 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:57.710552    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:27:57.765807 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:57.765771    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:27:59.521701 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:27:59.521665    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:01.217656 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:01.217624    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:02.920095 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:02.920064    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:04.390065 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:04.390025    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:04.709673 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:04.709584    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:06.424815 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:06.424782    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:08.022774 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:08.022736    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:08.858674 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:08.858639    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:28:09.723257 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:09.723220    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:11.475526 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:11.475490    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:12.615108 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:12.615070    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:13.307577 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:13.307536    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:14.607324 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:14.607295    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"]
Oct 31 21:28:14.612718 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:14.612700    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:28:14.631549 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:14.631524    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-cc45a7\"/\"openshift-service-ca.crt\""
Oct 31 21:28:14.631676 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:14.631591    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-cc45a7\"/\"kube-root-ca.crt\""
Oct 31 21:28:14.645490 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:14.645460    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-cc45a7\"/\"default-dockercfg-v4bmc\""
Oct 31 21:28:14.647794 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:14.647765    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"]
Oct 31 21:28:14.651987 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:14.651966    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-cc45a7\"/\"premium-w4-cc45a7-kserve-self-signed-certs\""
Oct 31 21:28:14.709406 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:14.709374    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-dshm\") pod \"premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt\" (UID: \"b9cd1fa6-02c8-42af-b733-8f9c326a5b73\") " pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:28:14.709570 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:14.709413    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-tmp-dir\") pod \"premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt\" (UID: \"b9cd1fa6-02c8-42af-b733-8f9c326a5b73\") " pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:28:14.709570 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:14.709543    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t9jx\" (UniqueName: \"kubernetes.io/projected/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-kube-api-access-6t9jx\") pod \"premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt\" (UID: \"b9cd1fa6-02c8-42af-b733-8f9c326a5b73\") " pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:28:14.709652 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:14.709596    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-tls-certs\") pod \"premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt\" (UID: \"b9cd1fa6-02c8-42af-b733-8f9c326a5b73\") " pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:28:14.709652 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:14.709623    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-home\") pod \"premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt\" (UID: \"b9cd1fa6-02c8-42af-b733-8f9c326a5b73\") " pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:28:14.709745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:14.709732    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-model-cache\") pod \"premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt\" (UID: \"b9cd1fa6-02c8-42af-b733-8f9c326a5b73\") " pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:28:14.735817 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:14.735781    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:14.810769 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:14.810733    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t9jx\" (UniqueName: \"kubernetes.io/projected/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-kube-api-access-6t9jx\") pod \"premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt\" (UID: \"b9cd1fa6-02c8-42af-b733-8f9c326a5b73\") " pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:28:14.810966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:14.810805    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-tls-certs\") pod \"premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt\" (UID: \"b9cd1fa6-02c8-42af-b733-8f9c326a5b73\") " pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:28:14.810966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:14.810834    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-home\") pod \"premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt\" (UID: \"b9cd1fa6-02c8-42af-b733-8f9c326a5b73\") " pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:28:14.810966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:14.810884    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-model-cache\") pod \"premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt\" (UID: \"b9cd1fa6-02c8-42af-b733-8f9c326a5b73\") " pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:28:14.810966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:14.810926    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-dshm\") pod \"premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt\" (UID: \"b9cd1fa6-02c8-42af-b733-8f9c326a5b73\") " pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:28:14.811220 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:14.810999    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-tmp-dir\") pod \"premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt\" (UID: \"b9cd1fa6-02c8-42af-b733-8f9c326a5b73\") " pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:28:14.811339 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:14.811301    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-home\") pod \"premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt\" (UID: \"b9cd1fa6-02c8-42af-b733-8f9c326a5b73\") " pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:28:14.811459 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:14.811349    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-model-cache\") pod \"premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt\" (UID: \"b9cd1fa6-02c8-42af-b733-8f9c326a5b73\") " pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:28:14.811526 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:14.811458    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-tmp-dir\") pod \"premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt\" (UID: \"b9cd1fa6-02c8-42af-b733-8f9c326a5b73\") " pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:28:14.813411 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:14.813392    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-tls-certs\") pod \"premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt\" (UID: \"b9cd1fa6-02c8-42af-b733-8f9c326a5b73\") " pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:28:14.813524 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:14.813505    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-dshm\") pod \"premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt\" (UID: \"b9cd1fa6-02c8-42af-b733-8f9c326a5b73\") " pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:28:14.851489 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:14.851461    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t9jx\" (UniqueName: \"kubernetes.io/projected/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-kube-api-access-6t9jx\") pod \"premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt\" (UID: \"b9cd1fa6-02c8-42af-b733-8f9c326a5b73\") " pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:28:14.925200 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:14.925123    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:28:15.225627 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:15.225600    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"]
Oct 31 21:28:15.226436 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:28:15.226411    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb9cd1fa6_02c8_42af_b733_8f9c326a5b73.slice/crio-7d5ef6f2dcaa5dc3dc0d02c96fe4eb0c54de4ec3c950a966ab9a0ef26f7f5cd6 WatchSource:0}: Error finding container 7d5ef6f2dcaa5dc3dc0d02c96fe4eb0c54de4ec3c950a966ab9a0ef26f7f5cd6: Status 404 returned error can't find the container with id 7d5ef6f2dcaa5dc3dc0d02c96fe4eb0c54de4ec3c950a966ab9a0ef26f7f5cd6
Oct 31 21:28:15.388477 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:15.388437    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:15.823518 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:15.823489    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" event={"ID":"b9cd1fa6-02c8-42af-b733-8f9c326a5b73","Type":"ContainerStarted","Data":"0779a636e28bb1df293e46a211d54a017014b80a7df74a50996120ea46660b22"}
Oct 31 21:28:15.823871 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:15.823526    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" event={"ID":"b9cd1fa6-02c8-42af-b733-8f9c326a5b73","Type":"ContainerStarted","Data":"7d5ef6f2dcaa5dc3dc0d02c96fe4eb0c54de4ec3c950a966ab9a0ef26f7f5cd6"}
Oct 31 21:28:15.878107 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:15.878059    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" podStartSLOduration=1.8780434860000002 podStartE2EDuration="1.878043486s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:28:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:28:15.876377295 +0000 UTC m=+735.792686726" watchObservedRunningTime="2026-10-31 21:28:15.878043486 +0000 UTC m=+735.794352913"
Oct 31 21:28:15.925357 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:15.925332    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:28:17.390150 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:17.390029    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:19.353632 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:19.353594    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:20.985952 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:20.985913    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:28:21.389674 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:21.389640    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:21.450746 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:28:21.450709    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:28:21.577626 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:21.577591    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:23.312892 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:23.312816    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:24.925470 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:24.925436    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:28:24.927207 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:24.927172    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:24.937667 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:24.937645    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:28:25.892573 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:25.892547    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:28:26.722019 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:26.721981    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:28.438866 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:28.438808    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:30.340950 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:30.340896    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:32.325613 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:32.325577    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:33.558196 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:33.558158    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"]
Oct 31 21:28:33.562254 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:33.562231    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 21:28:33.582742 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:33.582722    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-cc45a7-openshift-default-dockercfg-dbkx4\""
Oct 31 21:28:33.632624 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:33.632601    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"]
Oct 31 21:28:33.702926 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:33.702886    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-credential-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 21:28:33.703197 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:33.702959    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-istio-envoy\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 21:28:33.703197 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:33.703033    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-istio-data\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 21:28:33.703197 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:33.703072    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f60d565-50f1-4b9d-959a-014be6e49128-istiod-ca-cert\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 21:28:33.703197 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:33.703167    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s85wb\" (UniqueName: \"kubernetes.io/projected/3f60d565-50f1-4b9d-959a-014be6e49128-kube-api-access-s85wb\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 21:28:33.703197 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:33.703191    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-workload-certs\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 21:28:33.703506 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:33.703217    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-workload-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 21:28:33.703506 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:33.703254    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f60d565-50f1-4b9d-959a-014be6e49128-istio-podinfo\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 21:28:33.703506 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:33.703325    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f60d565-50f1-4b9d-959a-014be6e49128-istio-token\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 21:28:33.804487 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:33.804453    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-credential-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 21:28:33.804662 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:33.804546    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-istio-envoy\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 21:28:33.804662 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:33.804581    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-istio-data\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 21:28:33.804662 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:33.804624    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f60d565-50f1-4b9d-959a-014be6e49128-istiod-ca-cert\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 21:28:33.804823 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:33.804733    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s85wb\" (UniqueName: \"kubernetes.io/projected/3f60d565-50f1-4b9d-959a-014be6e49128-kube-api-access-s85wb\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 21:28:33.804823 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:33.804768    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-workload-certs\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 21:28:33.804823 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:33.804807    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-workload-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 21:28:33.804998 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:33.804834    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f60d565-50f1-4b9d-959a-014be6e49128-istio-podinfo\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 21:28:33.804998 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:33.804878    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f60d565-50f1-4b9d-959a-014be6e49128-istio-token\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 21:28:33.804998 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:33.804892    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-credential-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 21:28:33.804998 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:33.804931    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-istio-data\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 21:28:33.805272 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:33.805244    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-workload-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 21:28:33.805381 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:33.805357    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-workload-certs\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 21:28:33.805449 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:33.805377    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f60d565-50f1-4b9d-959a-014be6e49128-istiod-ca-cert\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 21:28:33.807102 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:33.807081    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-istio-envoy\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 21:28:33.807318 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:33.807296    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f60d565-50f1-4b9d-959a-014be6e49128-istio-podinfo\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 21:28:33.830069 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:33.830016    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s85wb\" (UniqueName: \"kubernetes.io/projected/3f60d565-50f1-4b9d-959a-014be6e49128-kube-api-access-s85wb\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 21:28:33.830152 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:33.830074    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f60d565-50f1-4b9d-959a-014be6e49128-istio-token\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 21:28:33.876515 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:33.876495    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 21:28:34.221317 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:28:34.221284    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f60d565_50f1_4b9d_959a_014be6e49128.slice/crio-0fce095fd23fd5b4dcc980c0481af897bf1664a19edc67e509e5c796ae47d9b3 WatchSource:0}: Error finding container 0fce095fd23fd5b4dcc980c0481af897bf1664a19edc67e509e5c796ae47d9b3: Status 404 returned error can't find the container with id 0fce095fd23fd5b4dcc980c0481af897bf1664a19edc67e509e5c796ae47d9b3
Oct 31 21:28:34.222382 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:34.222357    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:34.223917 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:34.223891    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:28:34.224018 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:34.223978    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:28:34.224018 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:34.224008    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:28:34.228348 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:34.228322    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"]
Oct 31 21:28:34.965620 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:34.965587    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk" event={"ID":"3f60d565-50f1-4b9d-959a-014be6e49128","Type":"ContainerStarted","Data":"b3d79e8dcda1d1ffba9095e19db1120efec82491a5488ddb2692667aedcb0ade"}
Oct 31 21:28:34.966043 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:34.965635    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk" event={"ID":"3f60d565-50f1-4b9d-959a-014be6e49128","Type":"ContainerStarted","Data":"0fce095fd23fd5b4dcc980c0481af897bf1664a19edc67e509e5c796ae47d9b3"}
Oct 31 21:28:35.046995 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:35.046907    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk" podStartSLOduration=2.046886758 podStartE2EDuration="2.046886758s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:28:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:28:35.044474924 +0000 UTC m=+754.960784353" watchObservedRunningTime="2026-10-31 21:28:35.046886758 +0000 UTC m=+754.963196187"
Oct 31 21:28:35.877335 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:35.877301    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 21:28:35.878803 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:35.878775    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 31 21:28:35.878933 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:35.878833    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk" podUID="3f60d565-50f1-4b9d-959a-014be6e49128" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 31 21:28:36.016118 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:36.016080    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:36.877396 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:36.877361    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 31 21:28:36.877576 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:36.877422    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk" podUID="3f60d565-50f1-4b9d-959a-014be6e49128" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 31 21:28:37.845565 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:37.845526    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:37.878960 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:37.876904    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 31 21:28:37.878960 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:37.877002    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk" podUID="3f60d565-50f1-4b9d-959a-014be6e49128" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 31 21:28:38.183003 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:38.182897    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:28:38.877587 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:38.877555    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 31 21:28:38.878030 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:38.877610    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk" podUID="3f60d565-50f1-4b9d-959a-014be6e49128" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 31 21:28:39.626960 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:39.620249    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:39.877662 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:39.877583    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Oct 31 21:28:39.877662 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:39.877634    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk" podUID="3f60d565-50f1-4b9d-959a-014be6e49128" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Oct 31 21:28:41.277013 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:41.276973    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 21:28:41.277526 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:41.277358    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 21:28:41.278329 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:41.278308    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 21:28:41.520398 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:41.520362    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:43.364090 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:43.364009    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:44.439581 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:44.439549    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:45.204599 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:45.204564    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:47.199869 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:47.199832    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:49.233681 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:49.233645    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:50.913000 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:50.912926    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:51.451557 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:28:51.451519    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:28:52.523781 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:52.523746    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:54.425668 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:54.425635    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:56.026705 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:56.026671    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:57.065972 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:57.065927    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:28:57.726913 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:57.726879    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:28:59.347084 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:28:59.347046    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:01.129218 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:01.129178    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:02.933277 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:02.933239    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:04.541908 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:04.541867    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:06.482899 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:06.482867    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:08.068742 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.068703    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"]
Oct 31 21:29:08.073221 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.073193    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:29:08.089912 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.089882    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-994336-openshift-default-dockercfg-qgxk4\""
Oct 31 21:29:08.100164 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.100138    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-workload-certs\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:29:08.100299 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.100190    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/845e8f3e-1896-4417-bfe3-5d1638925a57-istio-podinfo\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:29:08.100299 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.100226    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/845e8f3e-1896-4417-bfe3-5d1638925a57-istiod-ca-cert\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:29:08.100413 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.100355    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-workload-socket\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:29:08.100459 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.100418    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/845e8f3e-1896-4417-bfe3-5d1638925a57-istio-token\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:29:08.100507 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.100478    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-istio-data\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:29:08.100570 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.100554    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-credential-socket\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:29:08.100622 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.100607    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-istio-envoy\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:29:08.100680 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.100638    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lv2fv\" (UniqueName: \"kubernetes.io/projected/845e8f3e-1896-4417-bfe3-5d1638925a57-kube-api-access-lv2fv\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:29:08.106240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.106218    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"]
Oct 31 21:29:08.201571 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.201536    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-workload-certs\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:29:08.201749 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.201579    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/845e8f3e-1896-4417-bfe3-5d1638925a57-istio-podinfo\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:29:08.201749 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.201600    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/845e8f3e-1896-4417-bfe3-5d1638925a57-istiod-ca-cert\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:29:08.201749 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.201625    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-workload-socket\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:29:08.201749 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.201654    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/845e8f3e-1896-4417-bfe3-5d1638925a57-istio-token\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:29:08.201749 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.201704    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-istio-data\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:29:08.202089 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.201762    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-credential-socket\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:29:08.202089 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.201805    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-istio-envoy\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:29:08.202089 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.201828    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lv2fv\" (UniqueName: \"kubernetes.io/projected/845e8f3e-1896-4417-bfe3-5d1638925a57-kube-api-access-lv2fv\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:29:08.202089 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.202007    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-workload-certs\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:29:08.202291 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.202237    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-workload-socket\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:29:08.202335 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.202290    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-istio-data\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:29:08.202335 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.202326    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-credential-socket\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:29:08.202498 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.202473    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/845e8f3e-1896-4417-bfe3-5d1638925a57-istiod-ca-cert\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:29:08.204039 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.204018    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-istio-envoy\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:29:08.204100 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.204041    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/845e8f3e-1896-4417-bfe3-5d1638925a57-istio-podinfo\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:29:08.226838 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.226807    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/845e8f3e-1896-4417-bfe3-5d1638925a57-istio-token\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:29:08.227114 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.227092    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lv2fv\" (UniqueName: \"kubernetes.io/projected/845e8f3e-1896-4417-bfe3-5d1638925a57-kube-api-access-lv2fv\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:29:08.236230 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.236203    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:08.388136 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.388046    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:29:08.701629 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.701605    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"]
Oct 31 21:29:08.703299 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:29:08.703270    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod845e8f3e_1896_4417_bfe3_5d1638925a57.slice/crio-b590cd40663939f0a087cd66ef85d50c6610801590ad02ded1f317ec9b0102e6 WatchSource:0}: Error finding container b590cd40663939f0a087cd66ef85d50c6610801590ad02ded1f317ec9b0102e6: Status 404 returned error can't find the container with id b590cd40663939f0a087cd66ef85d50c6610801590ad02ded1f317ec9b0102e6
Oct 31 21:29:08.705468 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.705439    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:29:08.705527 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.705500    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:29:08.705569 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:08.705532    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:29:09.158202 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:09.158164    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m" event={"ID":"845e8f3e-1896-4417-bfe3-5d1638925a57","Type":"ContainerStarted","Data":"27c3d84586b0f096b6eed435b3db4a574ffba34a75a0abd502e9f8cdf370bbbd"}
Oct 31 21:29:09.158597 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:09.158220    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m" event={"ID":"845e8f3e-1896-4417-bfe3-5d1638925a57","Type":"ContainerStarted","Data":"b590cd40663939f0a087cd66ef85d50c6610801590ad02ded1f317ec9b0102e6"}
Oct 31 21:29:09.388744 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:09.388709    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:29:09.389934 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:09.389908    4460 patch_prober.go:29] interesting pod/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 31 21:29:09.390075 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:09.389984    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m" podUID="845e8f3e-1896-4417-bfe3-5d1638925a57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 31 21:29:10.053344 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:10.053305    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:10.389332 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:10.389249    4460 patch_prober.go:29] interesting pod/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 31 21:29:10.389332 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:10.389296    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m" podUID="845e8f3e-1896-4417-bfe3-5d1638925a57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 31 21:29:11.388885 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:11.388847    4460 patch_prober.go:29] interesting pod/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 31 21:29:11.389066 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:11.388917    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m" podUID="845e8f3e-1896-4417-bfe3-5d1638925a57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 31 21:29:11.746444 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:11.746359    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:12.388797 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:12.388757    4460 patch_prober.go:29] interesting pod/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 31 21:29:12.389024 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:12.388828    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m" podUID="845e8f3e-1896-4417-bfe3-5d1638925a57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 31 21:29:12.741016 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:12.740408    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:12.854865 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:12.854831    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:13.388317 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:13.388277    4460 patch_prober.go:29] interesting pod/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 31 21:29:13.388516 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:13.388336    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m" podUID="845e8f3e-1896-4417-bfe3-5d1638925a57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 31 21:29:13.689983 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:13.689890    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:14.356482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:14.356449    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:14.388636 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:14.388601    4460 patch_prober.go:29] interesting pod/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Oct 31 21:29:14.388806 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:14.388655    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m" podUID="845e8f3e-1896-4417-bfe3-5d1638925a57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Oct 31 21:29:15.396212 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:15.396176    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:29:15.398143 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:15.398122    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:29:15.399525 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:15.399504    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:29:15.468959 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:15.467980    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m" podStartSLOduration=7.467961191 podStartE2EDuration="7.467961191s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:29:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:29:09.240264039 +0000 UTC m=+789.156573468" watchObservedRunningTime="2026-10-31 21:29:15.467961191 +0000 UTC m=+795.384270623"
Oct 31 21:29:16.057874 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:16.057839    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:17.645136 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:17.645082    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-94gsm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 21:29:17.645791 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:17.645757    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm" podUID="ae31c154-bb41-4ff0-aae3-93164dc757df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:29:17.653050 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:17.653019    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-hcd5s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 21:29:17.653242 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:17.653216    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s" podUID="c0272849-0715-4e09-8d0b-827116fb35d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:29:18.612793 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:18.612754    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:20.519165 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:20.519133    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:21.451205 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:29:21.451173    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:29:22.323296 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:22.321740    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:22.377454 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:22.377403    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:29:24.250240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:24.250208    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:26.241456 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:26.241424    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:27.436237 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:27.436189    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"]
Oct 31 21:29:27.440240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:27.440224    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:29:27.469221 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:27.469193    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"]
Oct 31 21:29:27.602209 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:27.602177    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpc8g\" (UniqueName: \"kubernetes.io/projected/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-kube-api-access-gpc8g\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:29:27.602378 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:27.602229    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istio-podinfo\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:29:27.602378 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:27.602327    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istiod-ca-cert\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:29:27.602490 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:27.602399    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istio-data\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:29:27.602552 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:27.602485    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-workload-socket\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:29:27.602552 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:27.602510    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istio-envoy\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:29:27.602655 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:27.602624    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-workload-certs\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:29:27.602705 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:27.602692    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-credential-socket\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:29:27.602749 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:27.602721    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istio-token\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:29:27.703616 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:27.703548    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-workload-socket\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:29:27.703616 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:27.703587    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istio-envoy\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:29:27.703814 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:27.703615    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-workload-certs\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:29:27.703814 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:27.703672    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-credential-socket\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:29:27.703814 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:27.703708    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istio-token\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:29:27.703814 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:27.703769    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpc8g\" (UniqueName: \"kubernetes.io/projected/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-kube-api-access-gpc8g\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:29:27.704053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:27.703811    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istio-podinfo\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:29:27.704053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:27.704005    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istiod-ca-cert\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:29:27.704154 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:27.704059    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istio-data\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:29:27.704154 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:27.704089    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-workload-certs\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:29:27.704229 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:27.704163    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-credential-socket\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:29:27.704283 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:27.704273    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-workload-socket\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:29:27.704375 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:27.704356    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istio-data\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:29:27.704732 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:27.704702    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istiod-ca-cert\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:29:27.706092 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:27.706071    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istio-envoy\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:29:27.706314 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:27.706294    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istio-podinfo\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:29:27.728724 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:27.728694    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istio-token\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:29:27.728989 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:27.728971    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpc8g\" (UniqueName: \"kubernetes.io/projected/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-kube-api-access-gpc8g\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:29:27.754012 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:27.753993    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:29:28.015307 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:28.015273    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:28.080056 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:28.079999    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"]
Oct 31 21:29:28.086347 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:29:28.086310    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00f0f1bb_f3ba_4eb0_9fba_147fb2e3d50a.slice/crio-be22e7774a11b8d18b70095a0a169a288043d0892a07eb7f6f784377b1c8e160 WatchSource:0}: Error finding container be22e7774a11b8d18b70095a0a169a288043d0892a07eb7f6f784377b1c8e160: Status 404 returned error can't find the container with id be22e7774a11b8d18b70095a0a169a288043d0892a07eb7f6f784377b1c8e160
Oct 31 21:29:28.089405 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:28.089364    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:29:28.089534 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:28.089463    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:29:28.089534 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:28.089515    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:29:28.344600 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:28.344510    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659" event={"ID":"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a","Type":"ContainerStarted","Data":"7dea388ed0cfdce2384608dd6431fbe8aeb85ad4504ad1152509d16d97531a8d"}
Oct 31 21:29:28.344600 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:28.344578    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659" event={"ID":"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a","Type":"ContainerStarted","Data":"be22e7774a11b8d18b70095a0a169a288043d0892a07eb7f6f784377b1c8e160"}
Oct 31 21:29:28.394062 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:28.394013    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659" podStartSLOduration=1.393999174 podStartE2EDuration="1.393999174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:29:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:29:28.39326174 +0000 UTC m=+808.309571169" watchObservedRunningTime="2026-10-31 21:29:28.393999174 +0000 UTC m=+808.310308602"
Oct 31 21:29:28.754803 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:28.754701    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:29:29.756411 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:29.756376    4460 patch_prober.go:29] interesting pod/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 31 21:29:29.756806 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:29.756438    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659" podUID="00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 31 21:29:29.789879 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:29.789849    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:30.755391 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:30.755349    4460 patch_prober.go:29] interesting pod/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 31 21:29:30.755699 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:30.755664    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659" podUID="00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 31 21:29:31.757376 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:31.757337    4460 patch_prober.go:29] interesting pod/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 31 21:29:31.758074 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:31.758037    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659" podUID="00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 31 21:29:31.833777 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:31.833685    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:32.652114 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:32.652075    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-94gsm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 21:29:32.652421 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:32.652372    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm" podUID="ae31c154-bb41-4ff0-aae3-93164dc757df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:29:32.658061 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:32.658019    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-hcd5s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 21:29:32.658183 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:32.658067    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s" podUID="c0272849-0715-4e09-8d0b-827116fb35d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:29:32.759520 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:32.759481    4460 patch_prober.go:29] interesting pod/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 31 21:29:32.760134 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:32.760096    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659" podUID="00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 31 21:29:33.754472 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:33.754436    4460 patch_prober.go:29] interesting pod/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Oct 31 21:29:33.754629 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:33.754495    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659" podUID="00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Oct 31 21:29:33.846565 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:33.846523    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:34.046541 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:34.046499    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:35.064359 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:35.064324    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:29:35.064874 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:35.064854    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:29:35.065663 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:35.065635    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:29:35.556958 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:35.556889    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:37.370074 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:37.370037    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:39.124527 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:39.124493    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:40.891995 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:40.891957    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:40.944168 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:40.944115    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:29:42.952884 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:42.952845    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:44.668325 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:44.668287    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:45.230084 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:45.230053    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:47.132992 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:47.132958    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:47.843914 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:47.843883    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"]
Oct 31 21:29:47.847709 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:47.847688    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:29:47.864653 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:47.864635    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-c16684-openshift-default-dockercfg-t7nbj\""
Oct 31 21:29:47.881517 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:47.881495    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"]
Oct 31 21:29:48.009884 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.009855    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istio-podinfo\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:29:48.010088 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.009892    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istio-token\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:29:48.010088 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.009915    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-workload-certs\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:29:48.010088 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.010013    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istio-data\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:29:48.010222 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.010126    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lp959\" (UniqueName: \"kubernetes.io/projected/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-kube-api-access-lp959\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:29:48.010222 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.010182    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-workload-socket\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:29:48.010281 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.010263    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-credential-socket\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:29:48.010352 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.010339    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istiod-ca-cert\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:29:48.010389 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.010372    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istio-envoy\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:29:48.111111 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.111042    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istiod-ca-cert\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:29:48.111111 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.111081    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istio-envoy\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:29:48.111111 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.111105    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istio-podinfo\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:29:48.111368 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.111125    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istio-token\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:29:48.111368 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.111145    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-workload-certs\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:29:48.111368 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.111260    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istio-data\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:29:48.111368 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.111354    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lp959\" (UniqueName: \"kubernetes.io/projected/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-kube-api-access-lp959\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:29:48.111601 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.111390    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-workload-socket\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:29:48.111601 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.111488    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-credential-socket\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:29:48.111601 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.111509    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-workload-certs\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:29:48.111601 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.111595    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istio-data\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:29:48.111861 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.111836    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-workload-socket\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:29:48.111930 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.111893    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-credential-socket\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:29:48.112100 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.112081    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istiod-ca-cert\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:29:48.113450 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.113422    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istio-envoy\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:29:48.113835 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.113815    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istio-podinfo\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:29:48.139392 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.139364    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istio-token\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:29:48.139673 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.139536    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lp959\" (UniqueName: \"kubernetes.io/projected/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-kube-api-access-lp959\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:29:48.161225 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.161202    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:29:48.477109 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.477079    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"]
Oct 31 21:29:48.478578 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:29:48.478548    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9410a6ce_b00f_4f8e_bdba_7a01cc4f4f9e.slice/crio-f4281304c5f0e90a6a3b409a9360282f35c4d1e00f6633ae7d1a9d5de9a3f312 WatchSource:0}: Error finding container f4281304c5f0e90a6a3b409a9360282f35c4d1e00f6633ae7d1a9d5de9a3f312: Status 404 returned error can't find the container with id f4281304c5f0e90a6a3b409a9360282f35c4d1e00f6633ae7d1a9d5de9a3f312
Oct 31 21:29:48.480846 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.480809    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:29:48.480995 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.480887    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:29:48.480995 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.480928    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:29:48.538484 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.538459    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc" event={"ID":"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e","Type":"ContainerStarted","Data":"f4281304c5f0e90a6a3b409a9360282f35c4d1e00f6633ae7d1a9d5de9a3f312"}
Oct 31 21:29:48.824686 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:48.824653    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:49.552775 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:49.552736    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc" event={"ID":"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e","Type":"ContainerStarted","Data":"d4ad876cc03acf1f00d975c0660392bbf997eafbe355d59881fb056790c58be6"}
Oct 31 21:29:49.628961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:49.620597    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc" podStartSLOduration=2.620579392 podStartE2EDuration="2.620579392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:29:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:29:49.618030371 +0000 UTC m=+829.534339798" watchObservedRunningTime="2026-10-31 21:29:49.620579392 +0000 UTC m=+829.536888821"
Oct 31 21:29:50.161647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:50.161611    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:29:50.162995 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:50.162964    4460 patch_prober.go:29] interesting pod/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Oct 31 21:29:50.163119 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:50.163017    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc" podUID="9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Oct 31 21:29:50.354102 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:50.354059    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:50.487012 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:50.484789    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:51.162456 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:51.162417    4460 patch_prober.go:29] interesting pod/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Oct 31 21:29:51.163065 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:51.163018    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc" podUID="9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Oct 31 21:29:51.468668 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:29:51.468590    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:29:51.846437 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:51.846406    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:52.161801 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:52.161719    4460 patch_prober.go:29] interesting pod/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Oct 31 21:29:52.161801 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:52.161776    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc" podUID="9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Oct 31 21:29:52.345318 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:52.345276    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:53.162435 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:53.162400    4460 patch_prober.go:29] interesting pod/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Oct 31 21:29:53.162614 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:53.162455    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc" podUID="9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Oct 31 21:29:54.161642 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:54.161610    4460 patch_prober.go:29] interesting pod/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Oct 31 21:29:54.162033 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:54.161664    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc" podUID="9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Oct 31 21:29:54.232312 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:54.232273    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:55.165152 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:55.165121    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:29:55.165538 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:55.165513    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:29:55.166213 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:55.166194    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:29:56.026902 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:56.026866    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:29:57.484140 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:57.484105    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:29:57.818341 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:29:57.818307    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:00.398324 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:00.398255    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:02.952199 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:02.952165    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:04.372264 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:04.372221    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2"]
Oct 31 21:30:04.377504 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:04.377483    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2"
Oct 31 21:30:04.399416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:04.399390    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2"]
Oct 31 21:30:04.410432 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:04.410402    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-106555\"/\"e2e-conflict-b-08e7b19c-kserve-self-signed-certs\""
Oct 31 21:30:04.410558 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:04.410509    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-106555\"/\"default-dockercfg-g8p4t\""
Oct 31 21:30:04.414263 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:04.414243    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-106555\"/\"kube-root-ca.crt\""
Oct 31 21:30:04.414365 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:04.414284    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-106555\"/\"openshift-service-ca.crt\""
Oct 31 21:30:04.440852 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:04.440828    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:04.514512 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:04.514476    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8hfc\" (UniqueName: \"kubernetes.io/projected/3709f63c-b3c4-48c3-ab31-5ce42a565c72-kube-api-access-c8hfc\") pod \"e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2\" (UID: \"3709f63c-b3c4-48c3-ab31-5ce42a565c72\") " pod="e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2"
Oct 31 21:30:04.514512 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:04.514511    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3709f63c-b3c4-48c3-ab31-5ce42a565c72-home\") pod \"e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2\" (UID: \"3709f63c-b3c4-48c3-ab31-5ce42a565c72\") " pod="e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2"
Oct 31 21:30:04.514721 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:04.514663    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3709f63c-b3c4-48c3-ab31-5ce42a565c72-tls-certs\") pod \"e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2\" (UID: \"3709f63c-b3c4-48c3-ab31-5ce42a565c72\") " pod="e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2"
Oct 31 21:30:04.514721 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:04.514688    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3709f63c-b3c4-48c3-ab31-5ce42a565c72-tmp-dir\") pod \"e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2\" (UID: \"3709f63c-b3c4-48c3-ab31-5ce42a565c72\") " pod="e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2"
Oct 31 21:30:04.514721 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:04.514702    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3709f63c-b3c4-48c3-ab31-5ce42a565c72-dshm\") pod \"e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2\" (UID: \"3709f63c-b3c4-48c3-ab31-5ce42a565c72\") " pod="e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2"
Oct 31 21:30:04.514889 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:04.514845    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3709f63c-b3c4-48c3-ab31-5ce42a565c72-model-cache\") pod \"e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2\" (UID: \"3709f63c-b3c4-48c3-ab31-5ce42a565c72\") " pod="e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2"
Oct 31 21:30:04.615777 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:04.615749    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3709f63c-b3c4-48c3-ab31-5ce42a565c72-tls-certs\") pod \"e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2\" (UID: \"3709f63c-b3c4-48c3-ab31-5ce42a565c72\") " pod="e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2"
Oct 31 21:30:04.615912 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:04.615789    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3709f63c-b3c4-48c3-ab31-5ce42a565c72-tmp-dir\") pod \"e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2\" (UID: \"3709f63c-b3c4-48c3-ab31-5ce42a565c72\") " pod="e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2"
Oct 31 21:30:04.615912 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:04.615804    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3709f63c-b3c4-48c3-ab31-5ce42a565c72-dshm\") pod \"e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2\" (UID: \"3709f63c-b3c4-48c3-ab31-5ce42a565c72\") " pod="e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2"
Oct 31 21:30:04.615912 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:04.615863    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3709f63c-b3c4-48c3-ab31-5ce42a565c72-model-cache\") pod \"e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2\" (UID: \"3709f63c-b3c4-48c3-ab31-5ce42a565c72\") " pod="e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2"
Oct 31 21:30:04.616089 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:04.615911    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8hfc\" (UniqueName: \"kubernetes.io/projected/3709f63c-b3c4-48c3-ab31-5ce42a565c72-kube-api-access-c8hfc\") pod \"e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2\" (UID: \"3709f63c-b3c4-48c3-ab31-5ce42a565c72\") " pod="e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2"
Oct 31 21:30:04.616089 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:04.615967    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3709f63c-b3c4-48c3-ab31-5ce42a565c72-home\") pod \"e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2\" (UID: \"3709f63c-b3c4-48c3-ab31-5ce42a565c72\") " pod="e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2"
Oct 31 21:30:04.616251 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:04.616223    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3709f63c-b3c4-48c3-ab31-5ce42a565c72-tmp-dir\") pod \"e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2\" (UID: \"3709f63c-b3c4-48c3-ab31-5ce42a565c72\") " pod="e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2"
Oct 31 21:30:04.616305 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:04.616250    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3709f63c-b3c4-48c3-ab31-5ce42a565c72-model-cache\") pod \"e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2\" (UID: \"3709f63c-b3c4-48c3-ab31-5ce42a565c72\") " pod="e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2"
Oct 31 21:30:04.616380 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:04.616368    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3709f63c-b3c4-48c3-ab31-5ce42a565c72-home\") pod \"e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2\" (UID: \"3709f63c-b3c4-48c3-ab31-5ce42a565c72\") " pod="e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2"
Oct 31 21:30:04.617987 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:04.617968    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3709f63c-b3c4-48c3-ab31-5ce42a565c72-dshm\") pod \"e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2\" (UID: \"3709f63c-b3c4-48c3-ab31-5ce42a565c72\") " pod="e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2"
Oct 31 21:30:04.618278 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:04.618263    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3709f63c-b3c4-48c3-ab31-5ce42a565c72-tls-certs\") pod \"e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2\" (UID: \"3709f63c-b3c4-48c3-ab31-5ce42a565c72\") " pod="e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2"
Oct 31 21:30:04.642642 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:04.642585    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8hfc\" (UniqueName: \"kubernetes.io/projected/3709f63c-b3c4-48c3-ab31-5ce42a565c72-kube-api-access-c8hfc\") pod \"e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2\" (UID: \"3709f63c-b3c4-48c3-ab31-5ce42a565c72\") " pod="e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2"
Oct 31 21:30:04.674047 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:04.674024    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:04.690484 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:04.690464    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2"
Oct 31 21:30:05.072019 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:05.071991    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2"]
Oct 31 21:30:05.073584 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:30:05.073563    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3709f63c_b3c4_48c3_ab31_5ce42a565c72.slice/crio-15c91d162e75512d854a44ea1e5365ea6a7ba22bf58c2e17e37bf3275e880880 WatchSource:0}: Error finding container 15c91d162e75512d854a44ea1e5365ea6a7ba22bf58c2e17e37bf3275e880880: Status 404 returned error can't find the container with id 15c91d162e75512d854a44ea1e5365ea6a7ba22bf58c2e17e37bf3275e880880
Oct 31 21:30:05.777567 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:05.777526    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2" event={"ID":"3709f63c-b3c4-48c3-ab31-5ce42a565c72","Type":"ContainerStarted","Data":"b7467f28ff0a4a8b7f5c3cb8df998314e7ac2914dd0c54cfa57c70a394cfbee2"}
Oct 31 21:30:05.777979 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:05.777577    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2" event={"ID":"3709f63c-b3c4-48c3-ab31-5ce42a565c72","Type":"ContainerStarted","Data":"15c91d162e75512d854a44ea1e5365ea6a7ba22bf58c2e17e37bf3275e880880"}
Oct 31 21:30:05.828191 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:05.828146    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2" podStartSLOduration=1.8281322740000001 podStartE2EDuration="1.828132274s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:30:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:30:05.82781932 +0000 UTC m=+845.744128749" watchObservedRunningTime="2026-10-31 21:30:05.828132274 +0000 UTC m=+845.744441702"
Oct 31 21:30:06.442772 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:06.442731    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:06.690784 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:06.690745    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2"
Oct 31 21:30:06.795407 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:06.795370    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:08.226781 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:08.226733    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:09.941575 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:09.941537    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:11.214670 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:11.214634    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:11.331806 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:11.331769    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:11.784150 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:11.784119    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:13.556697 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:13.556594    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:14.691211 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:14.691179    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2"
Oct 31 21:30:14.703561 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:14.703535    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2"
Oct 31 21:30:14.838448 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:14.838419    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2"
Oct 31 21:30:15.516970 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:15.516922    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:17.328779 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:17.328741    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:19.152511 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:19.152475    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:20.413332 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:20.413299    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:30:20.952626 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:20.952588    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"]
Oct 31 21:30:20.957559 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:20.957538    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:30:20.978671 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:20.978631    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-4b5989\"/\"openshift-service-ca.crt\""
Oct 31 21:30:20.978867 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:20.978849    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"]
Oct 31 21:30:20.988008 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:20.987985    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-4b5989\"/\"distinct-2-w0-4b5989-kserve-self-signed-certs\""
Oct 31 21:30:20.988008 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:20.987994    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-4b5989\"/\"default-dockercfg-tdnws\""
Oct 31 21:30:20.988176 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:20.988004    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-4b5989\"/\"kube-root-ca.crt\""
Oct 31 21:30:21.008340 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:21.008307    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:21.026859 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:21.026833    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af7e0e0c-cf52-4270-b528-35718339996c-home\") pod \"distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274\" (UID: \"af7e0e0c-cf52-4270-b528-35718339996c\") " pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:30:21.027021 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:21.026884    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af7e0e0c-cf52-4270-b528-35718339996c-model-cache\") pod \"distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274\" (UID: \"af7e0e0c-cf52-4270-b528-35718339996c\") " pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:30:21.027021 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:21.026998    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af7e0e0c-cf52-4270-b528-35718339996c-tls-certs\") pod \"distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274\" (UID: \"af7e0e0c-cf52-4270-b528-35718339996c\") " pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:30:21.027119 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:21.027041    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af7e0e0c-cf52-4270-b528-35718339996c-tmp-dir\") pod \"distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274\" (UID: \"af7e0e0c-cf52-4270-b528-35718339996c\") " pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:30:21.027119 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:21.027068    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg8kr\" (UniqueName: \"kubernetes.io/projected/af7e0e0c-cf52-4270-b528-35718339996c-kube-api-access-kg8kr\") pod \"distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274\" (UID: \"af7e0e0c-cf52-4270-b528-35718339996c\") " pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:30:21.027211 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:21.027147    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af7e0e0c-cf52-4270-b528-35718339996c-dshm\") pod \"distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274\" (UID: \"af7e0e0c-cf52-4270-b528-35718339996c\") " pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:30:21.128777 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:21.128744    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af7e0e0c-cf52-4270-b528-35718339996c-model-cache\") pod \"distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274\" (UID: \"af7e0e0c-cf52-4270-b528-35718339996c\") " pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:30:21.128935 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:21.128805    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af7e0e0c-cf52-4270-b528-35718339996c-tls-certs\") pod \"distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274\" (UID: \"af7e0e0c-cf52-4270-b528-35718339996c\") " pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:30:21.128935 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:21.128835    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af7e0e0c-cf52-4270-b528-35718339996c-tmp-dir\") pod \"distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274\" (UID: \"af7e0e0c-cf52-4270-b528-35718339996c\") " pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:30:21.128935 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:21.128861    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg8kr\" (UniqueName: \"kubernetes.io/projected/af7e0e0c-cf52-4270-b528-35718339996c-kube-api-access-kg8kr\") pod \"distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274\" (UID: \"af7e0e0c-cf52-4270-b528-35718339996c\") " pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:30:21.128935 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:21.128887    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af7e0e0c-cf52-4270-b528-35718339996c-dshm\") pod \"distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274\" (UID: \"af7e0e0c-cf52-4270-b528-35718339996c\") " pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:30:21.128935 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:21.128964    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af7e0e0c-cf52-4270-b528-35718339996c-home\") pod \"distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274\" (UID: \"af7e0e0c-cf52-4270-b528-35718339996c\") " pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:30:21.129294 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:21.129273    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af7e0e0c-cf52-4270-b528-35718339996c-model-cache\") pod \"distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274\" (UID: \"af7e0e0c-cf52-4270-b528-35718339996c\") " pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:30:21.129363 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:21.129310    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af7e0e0c-cf52-4270-b528-35718339996c-tmp-dir\") pod \"distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274\" (UID: \"af7e0e0c-cf52-4270-b528-35718339996c\") " pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:30:21.129430 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:21.129401    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af7e0e0c-cf52-4270-b528-35718339996c-home\") pod \"distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274\" (UID: \"af7e0e0c-cf52-4270-b528-35718339996c\") " pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:30:21.131390 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:21.131362    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af7e0e0c-cf52-4270-b528-35718339996c-dshm\") pod \"distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274\" (UID: \"af7e0e0c-cf52-4270-b528-35718339996c\") " pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:30:21.131480 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:21.131462    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af7e0e0c-cf52-4270-b528-35718339996c-tls-certs\") pod \"distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274\" (UID: \"af7e0e0c-cf52-4270-b528-35718339996c\") " pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:30:21.158397 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:21.158376    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg8kr\" (UniqueName: \"kubernetes.io/projected/af7e0e0c-cf52-4270-b528-35718339996c-kube-api-access-kg8kr\") pod \"distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274\" (UID: \"af7e0e0c-cf52-4270-b528-35718339996c\") " pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:30:21.277960 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:21.272732    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:30:21.394803 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:21.394716    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:21.475098 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:30:21.475053    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:30:21.668363 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:21.668334    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"]
Oct 31 21:30:21.671544 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:30:21.671515    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaf7e0e0c_cf52_4270_b528_35718339996c.slice/crio-5158e2d12f6ff9616d3df107159a399424fe3f738dfa0c70a80721dff32b0fb7 WatchSource:0}: Error finding container 5158e2d12f6ff9616d3df107159a399424fe3f738dfa0c70a80721dff32b0fb7: Status 404 returned error can't find the container with id 5158e2d12f6ff9616d3df107159a399424fe3f738dfa0c70a80721dff32b0fb7
Oct 31 21:30:21.872844 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:21.872797    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" event={"ID":"af7e0e0c-cf52-4270-b528-35718339996c","Type":"ContainerStarted","Data":"dc13d5467c37ab13f3d4d85e18efb588620e00e8261274c81f0a724b8fc5049f"}
Oct 31 21:30:21.873046 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:21.872857    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" event={"ID":"af7e0e0c-cf52-4270-b528-35718339996c","Type":"ContainerStarted","Data":"5158e2d12f6ff9616d3df107159a399424fe3f738dfa0c70a80721dff32b0fb7"}
Oct 31 21:30:21.929821 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:21.929715    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" podStartSLOduration=1.929695682 podStartE2EDuration="1.929695682s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:30:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:30:21.926743979 +0000 UTC m=+861.843053408" watchObservedRunningTime="2026-10-31 21:30:21.929695682 +0000 UTC m=+861.846005112"
Oct 31 21:30:22.273303 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:22.273265    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:30:23.903424 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:23.903369    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:26.056480 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:26.056437    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:27.764786 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:27.764747    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:29.330355 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:29.330320    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"]
Oct 31 21:30:29.334669 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:29.334639    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:30:29.367301 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:29.367180    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-c16684\"/\"distinct-2-w2-c16684-kserve-self-signed-certs\""
Oct 31 21:30:29.367301 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:29.367183    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-c16684\"/\"default-dockercfg-lk94p\""
Oct 31 21:30:29.367511 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:29.367394    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-c16684\"/\"openshift-service-ca.crt\""
Oct 31 21:30:29.367511 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:29.367395    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-c16684\"/\"kube-root-ca.crt\""
Oct 31 21:30:29.367700 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:29.367679    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93a42128-ee74-4dcd-b158-27e33cfd44f9-model-cache\") pod \"distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz\" (UID: \"93a42128-ee74-4dcd-b158-27e33cfd44f9\") " pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:30:29.367767 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:29.367728    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93a42128-ee74-4dcd-b158-27e33cfd44f9-tmp-dir\") pod \"distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz\" (UID: \"93a42128-ee74-4dcd-b158-27e33cfd44f9\") " pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:30:29.367767 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:29.367756    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93a42128-ee74-4dcd-b158-27e33cfd44f9-home\") pod \"distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz\" (UID: \"93a42128-ee74-4dcd-b158-27e33cfd44f9\") " pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:30:29.367875 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:29.367861    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjh6f\" (UniqueName: \"kubernetes.io/projected/93a42128-ee74-4dcd-b158-27e33cfd44f9-kube-api-access-cjh6f\") pod \"distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz\" (UID: \"93a42128-ee74-4dcd-b158-27e33cfd44f9\") " pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:30:29.367962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:29.367931    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93a42128-ee74-4dcd-b158-27e33cfd44f9-tls-certs\") pod \"distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz\" (UID: \"93a42128-ee74-4dcd-b158-27e33cfd44f9\") " pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:30:29.368056 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:29.368040    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93a42128-ee74-4dcd-b158-27e33cfd44f9-dshm\") pod \"distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz\" (UID: \"93a42128-ee74-4dcd-b158-27e33cfd44f9\") " pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:30:29.371491 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:29.371468    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"]
Oct 31 21:30:29.392093 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:29.392067    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:29.468827 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:29.468800    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93a42128-ee74-4dcd-b158-27e33cfd44f9-dshm\") pod \"distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz\" (UID: \"93a42128-ee74-4dcd-b158-27e33cfd44f9\") " pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:30:29.468975 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:29.468926    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93a42128-ee74-4dcd-b158-27e33cfd44f9-model-cache\") pod \"distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz\" (UID: \"93a42128-ee74-4dcd-b158-27e33cfd44f9\") " pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:30:29.469042 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:29.468991    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93a42128-ee74-4dcd-b158-27e33cfd44f9-tmp-dir\") pod \"distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz\" (UID: \"93a42128-ee74-4dcd-b158-27e33cfd44f9\") " pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:30:29.469115 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:29.469093    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93a42128-ee74-4dcd-b158-27e33cfd44f9-home\") pod \"distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz\" (UID: \"93a42128-ee74-4dcd-b158-27e33cfd44f9\") " pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:30:29.469170 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:29.469151    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjh6f\" (UniqueName: \"kubernetes.io/projected/93a42128-ee74-4dcd-b158-27e33cfd44f9-kube-api-access-cjh6f\") pod \"distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz\" (UID: \"93a42128-ee74-4dcd-b158-27e33cfd44f9\") " pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:30:29.469236 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:29.469224    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93a42128-ee74-4dcd-b158-27e33cfd44f9-tls-certs\") pod \"distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz\" (UID: \"93a42128-ee74-4dcd-b158-27e33cfd44f9\") " pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:30:29.469375 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:29.469352    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93a42128-ee74-4dcd-b158-27e33cfd44f9-tmp-dir\") pod \"distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz\" (UID: \"93a42128-ee74-4dcd-b158-27e33cfd44f9\") " pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:30:29.469375 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:29.469366    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93a42128-ee74-4dcd-b158-27e33cfd44f9-model-cache\") pod \"distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz\" (UID: \"93a42128-ee74-4dcd-b158-27e33cfd44f9\") " pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:30:29.469529 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:29.469390    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93a42128-ee74-4dcd-b158-27e33cfd44f9-home\") pod \"distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz\" (UID: \"93a42128-ee74-4dcd-b158-27e33cfd44f9\") " pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:30:29.471024 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:29.471005    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93a42128-ee74-4dcd-b158-27e33cfd44f9-dshm\") pod \"distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz\" (UID: \"93a42128-ee74-4dcd-b158-27e33cfd44f9\") " pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:30:29.471505 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:29.471485    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93a42128-ee74-4dcd-b158-27e33cfd44f9-tls-certs\") pod \"distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz\" (UID: \"93a42128-ee74-4dcd-b158-27e33cfd44f9\") " pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:30:29.493856 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:29.493828    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjh6f\" (UniqueName: \"kubernetes.io/projected/93a42128-ee74-4dcd-b158-27e33cfd44f9-kube-api-access-cjh6f\") pod \"distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz\" (UID: \"93a42128-ee74-4dcd-b158-27e33cfd44f9\") " pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:30:29.517096 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:29.517072    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:29.648315 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:29.648176    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:30:29.954229 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:29.954202    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"]
Oct 31 21:30:29.955707 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:30:29.955684    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod93a42128_ee74_4dcd_b158_27e33cfd44f9.slice/crio-628f96f122bd76ec6ced13b47110682c74932e81e7b8ab70388403f798dd921b WatchSource:0}: Error finding container 628f96f122bd76ec6ced13b47110682c74932e81e7b8ab70388403f798dd921b: Status 404 returned error can't find the container with id 628f96f122bd76ec6ced13b47110682c74932e81e7b8ab70388403f798dd921b
Oct 31 21:30:30.946244 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:30.946207    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz" event={"ID":"93a42128-ee74-4dcd-b158-27e33cfd44f9","Type":"ContainerStarted","Data":"1ed2052e514627cb81374a78fd18e02589b6dffb1424d02405ab49435aa25ff6"}
Oct 31 21:30:30.946244 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:30.946250    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz" event={"ID":"93a42128-ee74-4dcd-b158-27e33cfd44f9","Type":"ContainerStarted","Data":"628f96f122bd76ec6ced13b47110682c74932e81e7b8ab70388403f798dd921b"}
Oct 31 21:30:30.992346 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:30.992296    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz" podStartSLOduration=1.992282388 podStartE2EDuration="1.992282388s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:30:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:30:30.990343989 +0000 UTC m=+870.906653419" watchObservedRunningTime="2026-10-31 21:30:30.992282388 +0000 UTC m=+870.908591815"
Oct 31 21:30:31.273439 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:31.273408    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:30:31.285666 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:31.285643    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:30:31.488152 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:31.488110    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:31.648982 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:31.648877    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:30:31.962835 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:31.962753    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:30:35.684690 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:35.684650    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:38.031441 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:38.031411    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2"]
Oct 31 21:30:38.066716 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:38.066686    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:38.910910 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:38.910873    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:39.648437 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:39.648395    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:30:39.660615 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:39.660593    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:30:40.041968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:40.041918    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:30:40.620015 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:40.619981    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:42.283408 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:42.283371    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:43.915063 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:43.915033    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:44.801167 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:44.801130    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:45.602404 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:45.602348    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:46.797870 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:46.797835    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:30:46.908980 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:46.908920    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:47.219135 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:47.219053    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:47.443533 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:47.443498    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:49.411105 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:49.411074    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:51.020617 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:51.020574    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:51.452601 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:30:51.452518    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:30:52.804865 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:52.804835    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:53.051150 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:53.051113    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2" podUID="3709f63c-b3c4-48c3-ab31-5ce42a565c72" containerName="main" containerID="cri-o://b7467f28ff0a4a8b7f5c3cb8df998314e7ac2914dd0c54cfa57c70a394cfbee2" gracePeriod=45
Oct 31 21:30:53.309059 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:53.309003    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2"
Oct 31 21:30:53.328400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:53.328376    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3709f63c-b3c4-48c3-ab31-5ce42a565c72-dshm\" (UniqueName: \"kubernetes.io/empty-dir/3709f63c-b3c4-48c3-ab31-5ce42a565c72-dshm\") pod \"3709f63c-b3c4-48c3-ab31-5ce42a565c72\" (UID: \"3709f63c-b3c4-48c3-ab31-5ce42a565c72\") "
Oct 31 21:30:53.328495 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:53.328426    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3709f63c-b3c4-48c3-ab31-5ce42a565c72-tls-certs\" (UniqueName: \"kubernetes.io/secret/3709f63c-b3c4-48c3-ab31-5ce42a565c72-tls-certs\") pod \"3709f63c-b3c4-48c3-ab31-5ce42a565c72\" (UID: \"3709f63c-b3c4-48c3-ab31-5ce42a565c72\") "
Oct 31 21:30:53.328495 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:53.328484    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3709f63c-b3c4-48c3-ab31-5ce42a565c72-kube-api-access-c8hfc\" (UniqueName: \"kubernetes.io/projected/3709f63c-b3c4-48c3-ab31-5ce42a565c72-kube-api-access-c8hfc\") pod \"3709f63c-b3c4-48c3-ab31-5ce42a565c72\" (UID: \"3709f63c-b3c4-48c3-ab31-5ce42a565c72\") "
Oct 31 21:30:53.328574 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:53.328513    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3709f63c-b3c4-48c3-ab31-5ce42a565c72-home\" (UniqueName: \"kubernetes.io/empty-dir/3709f63c-b3c4-48c3-ab31-5ce42a565c72-home\") pod \"3709f63c-b3c4-48c3-ab31-5ce42a565c72\" (UID: \"3709f63c-b3c4-48c3-ab31-5ce42a565c72\") "
Oct 31 21:30:53.328624 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:53.328608    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3709f63c-b3c4-48c3-ab31-5ce42a565c72-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3709f63c-b3c4-48c3-ab31-5ce42a565c72-model-cache\") pod \"3709f63c-b3c4-48c3-ab31-5ce42a565c72\" (UID: \"3709f63c-b3c4-48c3-ab31-5ce42a565c72\") "
Oct 31 21:30:53.328681 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:53.328666    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3709f63c-b3c4-48c3-ab31-5ce42a565c72-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3709f63c-b3c4-48c3-ab31-5ce42a565c72-tmp-dir\") pod \"3709f63c-b3c4-48c3-ab31-5ce42a565c72\" (UID: \"3709f63c-b3c4-48c3-ab31-5ce42a565c72\") "
Oct 31 21:30:53.328811 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:53.328786    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3709f63c-b3c4-48c3-ab31-5ce42a565c72-home" pod "3709f63c-b3c4-48c3-ab31-5ce42a565c72" (UID: "3709f63c-b3c4-48c3-ab31-5ce42a565c72"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:30:53.328968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:53.328903    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3709f63c-b3c4-48c3-ab31-5ce42a565c72-model-cache" pod "3709f63c-b3c4-48c3-ab31-5ce42a565c72" (UID: "3709f63c-b3c4-48c3-ab31-5ce42a565c72"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:30:53.329050 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:53.328982    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3709f63c-b3c4-48c3-ab31-5ce42a565c72-tmp-dir" pod "3709f63c-b3c4-48c3-ab31-5ce42a565c72" (UID: "3709f63c-b3c4-48c3-ab31-5ce42a565c72"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:30:53.329234 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:53.329215    4460 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3709f63c-b3c4-48c3-ab31-5ce42a565c72-home\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:30:53.329309 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:53.329238    4460 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3709f63c-b3c4-48c3-ab31-5ce42a565c72-model-cache\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:30:53.329309 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:53.329254    4460 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3709f63c-b3c4-48c3-ab31-5ce42a565c72-tmp-dir\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:30:53.330753 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:53.330733    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3709f63c-b3c4-48c3-ab31-5ce42a565c72-kube-api-access-c8hfc" pod "3709f63c-b3c4-48c3-ab31-5ce42a565c72" (UID: "3709f63c-b3c4-48c3-ab31-5ce42a565c72"). InnerVolumeSpecName "kube-api-access-c8hfc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:30:53.330860 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:53.330844    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3709f63c-b3c4-48c3-ab31-5ce42a565c72-dshm" pod "3709f63c-b3c4-48c3-ab31-5ce42a565c72" (UID: "3709f63c-b3c4-48c3-ab31-5ce42a565c72"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:30:53.330860 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:53.330850    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3709f63c-b3c4-48c3-ab31-5ce42a565c72-tls-certs" pod "3709f63c-b3c4-48c3-ab31-5ce42a565c72" (UID: "3709f63c-b3c4-48c3-ab31-5ce42a565c72"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 21:30:53.430893 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:53.430866    4460 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3709f63c-b3c4-48c3-ab31-5ce42a565c72-dshm\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:30:53.430893 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:53.430892    4460 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3709f63c-b3c4-48c3-ab31-5ce42a565c72-tls-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:30:53.431086 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:53.430904    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c8hfc\" (UniqueName: \"kubernetes.io/projected/3709f63c-b3c4-48c3-ab31-5ce42a565c72-kube-api-access-c8hfc\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:30:54.126500 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:54.126465    4460 generic.go:356] "Generic (PLEG): container finished" podID="3709f63c-b3c4-48c3-ab31-5ce42a565c72" containerID="b7467f28ff0a4a8b7f5c3cb8df998314e7ac2914dd0c54cfa57c70a394cfbee2" exitCode=0
Oct 31 21:30:54.127005 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:54.126520    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2"
Oct 31 21:30:54.127005 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:54.126546    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2" event={"ID":"3709f63c-b3c4-48c3-ab31-5ce42a565c72","Type":"ContainerDied","Data":"b7467f28ff0a4a8b7f5c3cb8df998314e7ac2914dd0c54cfa57c70a394cfbee2"}
Oct 31 21:30:54.127005 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:54.126597    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2" event={"ID":"3709f63c-b3c4-48c3-ab31-5ce42a565c72","Type":"ContainerDied","Data":"15c91d162e75512d854a44ea1e5365ea6a7ba22bf58c2e17e37bf3275e880880"}
Oct 31 21:30:54.127005 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:54.126623    4460 scope.go:122] "RemoveContainer" containerID="b7467f28ff0a4a8b7f5c3cb8df998314e7ac2914dd0c54cfa57c70a394cfbee2"
Oct 31 21:30:54.139912 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:54.139891    4460 scope.go:122] "RemoveContainer" containerID="b7467f28ff0a4a8b7f5c3cb8df998314e7ac2914dd0c54cfa57c70a394cfbee2"
Oct 31 21:30:54.140308 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:30:54.140280    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7467f28ff0a4a8b7f5c3cb8df998314e7ac2914dd0c54cfa57c70a394cfbee2\": container with ID starting with b7467f28ff0a4a8b7f5c3cb8df998314e7ac2914dd0c54cfa57c70a394cfbee2 not found: ID does not exist" containerID="b7467f28ff0a4a8b7f5c3cb8df998314e7ac2914dd0c54cfa57c70a394cfbee2"
Oct 31 21:30:54.140367 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:54.140319    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7467f28ff0a4a8b7f5c3cb8df998314e7ac2914dd0c54cfa57c70a394cfbee2"} err="failed to get container status \"b7467f28ff0a4a8b7f5c3cb8df998314e7ac2914dd0c54cfa57c70a394cfbee2\": rpc error: code = NotFound desc = could not find container \"b7467f28ff0a4a8b7f5c3cb8df998314e7ac2914dd0c54cfa57c70a394cfbee2\": container with ID starting with b7467f28ff0a4a8b7f5c3cb8df998314e7ac2914dd0c54cfa57c70a394cfbee2 not found: ID does not exist"
Oct 31 21:30:54.189204 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:54.189175    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2"]
Oct 31 21:30:54.202936 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:54.202914    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-106555/e2e-conflict-b-08e7b19c-kserve-65f94dfcfd-jvbb2"]
Oct 31 21:30:54.224350 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:54.224324    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3709f63c-b3c4-48c3-ab31-5ce42a565c72" path="/var/lib/kubelet/pods/3709f63c-b3c4-48c3-ab31-5ce42a565c72/volumes"
Oct 31 21:30:54.566870 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:54.566833    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:55.110753 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:55.110718    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:55.226423 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:55.226387    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:55.465767 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:55.465677    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-9888fbfd6-xfvf9"]
Oct 31 21:30:55.466307 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:55.466291    4460 cpu_manager.go:414] "containerMap: removing container" podUID="3709f63c-b3c4-48c3-ab31-5ce42a565c72" containerName="main"
Oct 31 21:30:55.466361 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:55.466308    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3709f63c-b3c4-48c3-ab31-5ce42a565c72" containerName="main"
Oct 31 21:30:55.466396 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:55.466383    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="3709f63c-b3c4-48c3-ab31-5ce42a565c72" containerName="main"
Oct 31 21:30:55.471028 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:55.471012    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-9888fbfd6-xfvf9"
Oct 31 21:30:55.501658 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:55.501626    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-0c70771b6656\""
Oct 31 21:30:55.501823 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:55.501688    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-mbzjk\""
Oct 31 21:30:55.501823 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:55.501730    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"routing-overlay\""
Oct 31 21:30:55.501823 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:55.501729    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"openshift-service-ca.crt\""
Oct 31 21:30:55.501823 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:55.501746    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-plugins\""
Oct 31 21:30:55.502196 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:55.502171    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"kube-root-ca.crt\""
Oct 31 21:30:55.504217 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:55.504196    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-9888fbfd6-xfvf9"]
Oct 31 21:30:55.558878 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:55.558847    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7llpm\" (UniqueName: \"kubernetes.io/projected/ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79-kube-api-access-7llpm\") pod \"payload-processing-external-model-9888fbfd6-xfvf9\" (UID: \"ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79\") " pod="models-as-a-service/payload-processing-external-model-9888fbfd6-xfvf9"
Oct 31 21:30:55.559032 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:55.558989    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79-provider-credentials\") pod \"payload-processing-external-model-9888fbfd6-xfvf9\" (UID: \"ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79\") " pod="models-as-a-service/payload-processing-external-model-9888fbfd6-xfvf9"
Oct 31 21:30:55.559096 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:55.559050    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79-routing-overlay-volume\") pod \"payload-processing-external-model-9888fbfd6-xfvf9\" (UID: \"ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79\") " pod="models-as-a-service/payload-processing-external-model-9888fbfd6-xfvf9"
Oct 31 21:30:55.559096 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:55.559082    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79-plugins-config-volume\") pod \"payload-processing-external-model-9888fbfd6-xfvf9\" (UID: \"ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79\") " pod="models-as-a-service/payload-processing-external-model-9888fbfd6-xfvf9"
Oct 31 21:30:55.659913 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:55.659872    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79-routing-overlay-volume\") pod \"payload-processing-external-model-9888fbfd6-xfvf9\" (UID: \"ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79\") " pod="models-as-a-service/payload-processing-external-model-9888fbfd6-xfvf9"
Oct 31 21:30:55.660122 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:55.659981    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79-plugins-config-volume\") pod \"payload-processing-external-model-9888fbfd6-xfvf9\" (UID: \"ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79\") " pod="models-as-a-service/payload-processing-external-model-9888fbfd6-xfvf9"
Oct 31 21:30:55.660122 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:55.660117    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7llpm\" (UniqueName: \"kubernetes.io/projected/ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79-kube-api-access-7llpm\") pod \"payload-processing-external-model-9888fbfd6-xfvf9\" (UID: \"ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79\") " pod="models-as-a-service/payload-processing-external-model-9888fbfd6-xfvf9"
Oct 31 21:30:55.660253 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:55.660234    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79-provider-credentials\") pod \"payload-processing-external-model-9888fbfd6-xfvf9\" (UID: \"ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79\") " pod="models-as-a-service/payload-processing-external-model-9888fbfd6-xfvf9"
Oct 31 21:30:55.660740 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:55.660718    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79-routing-overlay-volume\") pod \"payload-processing-external-model-9888fbfd6-xfvf9\" (UID: \"ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79\") " pod="models-as-a-service/payload-processing-external-model-9888fbfd6-xfvf9"
Oct 31 21:30:55.660969 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:55.660918    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79-plugins-config-volume\") pod \"payload-processing-external-model-9888fbfd6-xfvf9\" (UID: \"ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79\") " pod="models-as-a-service/payload-processing-external-model-9888fbfd6-xfvf9"
Oct 31 21:30:55.662709 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:55.662688    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79-provider-credentials\") pod \"payload-processing-external-model-9888fbfd6-xfvf9\" (UID: \"ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79\") " pod="models-as-a-service/payload-processing-external-model-9888fbfd6-xfvf9"
Oct 31 21:30:55.687071 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:55.687046    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7llpm\" (UniqueName: \"kubernetes.io/projected/ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79-kube-api-access-7llpm\") pod \"payload-processing-external-model-9888fbfd6-xfvf9\" (UID: \"ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79\") " pod="models-as-a-service/payload-processing-external-model-9888fbfd6-xfvf9"
Oct 31 21:30:55.785062 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:55.784995    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-9888fbfd6-xfvf9"
Oct 31 21:30:56.189642 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:56.189612    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-9888fbfd6-xfvf9"]
Oct 31 21:30:56.191573 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:30:56.191542    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae4c7e5e_3405_46aa_9f7f_faaef8a4cd79.slice/crio-a872d687c7b13648b656bc5a76ed51bd16387dc15c3b2123ea71faf12335cfcf WatchSource:0}: Error finding container a872d687c7b13648b656bc5a76ed51bd16387dc15c3b2123ea71faf12335cfcf: Status 404 returned error can't find the container with id a872d687c7b13648b656bc5a76ed51bd16387dc15c3b2123ea71faf12335cfcf
Oct 31 21:30:56.307039 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:56.306814    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:57.154073 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:57.154030    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-9888fbfd6-xfvf9" event={"ID":"ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79","Type":"ContainerStarted","Data":"e17718ccb01af32c5a6298ef762e08380e20f15e0cfdce9abff95e1945fff81c"}
Oct 31 21:30:57.154278 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:57.154115    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-9888fbfd6-xfvf9" event={"ID":"ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79","Type":"ContainerStarted","Data":"a872d687c7b13648b656bc5a76ed51bd16387dc15c3b2123ea71faf12335cfcf"}
Oct 31 21:30:57.154450 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:57.154428    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-9888fbfd6-xfvf9"
Oct 31 21:30:57.200514 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:57.200468    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-9888fbfd6-xfvf9" podStartSLOduration=2.20045316 podStartE2EDuration="2.20045316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:30:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:30:57.199641289 +0000 UTC m=+897.115950718" watchObservedRunningTime="2026-10-31 21:30:57.20045316 +0000 UTC m=+897.116762589"
Oct 31 21:30:58.458341 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:58.455951    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:30:58.880711 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:58.880665    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"]
Oct 31 21:30:58.884792 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:58.884769    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:30:58.913441 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:58.913416    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"]
Oct 31 21:30:58.934104 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:58.934070    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"]
Oct 31 21:30:58.942181 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:58.942152    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:30:58.965494 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:58.965451    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"]
Oct 31 21:30:59.029184 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.029146    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-credential-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:30:59.029365 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.029208    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8g6pr\" (UniqueName: \"kubernetes.io/projected/b06d985f-9a38-4ea8-a120-676d0c3de8aa-kube-api-access-8g6pr\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:30:59.029365 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.029277    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istio-envoy\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:30:59.029365 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.029309    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istio-data\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:30:59.029498 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.029380    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/475195a2-42a8-493d-ab68-585d187a977d-istio-podinfo\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:30:59.029498 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.029420    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istio-podinfo\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:30:59.029498 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.029439    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-istio-envoy\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:30:59.029498 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.029463    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-workload-certs\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:30:59.029654 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.029525    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/475195a2-42a8-493d-ab68-585d187a977d-istio-token\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:30:59.029654 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.029561    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istiod-ca-cert\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:30:59.029654 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.029644    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-credential-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:30:59.029765 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.029662    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bpkf\" (UniqueName: \"kubernetes.io/projected/475195a2-42a8-493d-ab68-585d187a977d-kube-api-access-2bpkf\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:30:59.029765 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.029690    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-workload-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:30:59.029829 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.029777    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istio-token\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:30:59.029829 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.029797    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/475195a2-42a8-493d-ab68-585d187a977d-istiod-ca-cert\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:30:59.029829 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.029812    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-workload-certs\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:30:59.029962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.029837    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-workload-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:30:59.029962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.029952    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-istio-data\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:30:59.130957 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.130852    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-istio-data\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:30:59.130957 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.130908    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-credential-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:30:59.131197 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.130966    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8g6pr\" (UniqueName: \"kubernetes.io/projected/b06d985f-9a38-4ea8-a120-676d0c3de8aa-kube-api-access-8g6pr\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:30:59.131197 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.130984    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istio-envoy\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:30:59.131197 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.131003    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istio-data\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:30:59.131197 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.131035    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/475195a2-42a8-493d-ab68-585d187a977d-istio-podinfo\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:30:59.131197 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.131065    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istio-podinfo\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:30:59.131197 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.131090    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-istio-envoy\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:30:59.131197 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.131145    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-workload-certs\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:30:59.131197 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.131177    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/475195a2-42a8-493d-ab68-585d187a977d-istio-token\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:30:59.131594 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.131201    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istiod-ca-cert\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:30:59.131594 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.131235    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-credential-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:30:59.131594 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.131259    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bpkf\" (UniqueName: \"kubernetes.io/projected/475195a2-42a8-493d-ab68-585d187a977d-kube-api-access-2bpkf\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:30:59.131594 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.131312    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-workload-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:30:59.131594 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.131377    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istio-token\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:30:59.131594 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.131415    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/475195a2-42a8-493d-ab68-585d187a977d-istiod-ca-cert\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:30:59.131594 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.131398    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-istio-data\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:30:59.131594 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.131440    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-workload-certs\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:30:59.131594 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.131506    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-workload-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:30:59.132062 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.131595    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istio-data\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:30:59.132062 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.131695    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-workload-certs\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:30:59.132062 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.131864    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-workload-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:30:59.132464 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.132127    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-credential-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:30:59.132464 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.132134    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istiod-ca-cert\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:30:59.132464 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.131429    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-credential-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:30:59.132464 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.131864    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-workload-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:30:59.132464 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.132411    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-workload-certs\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:30:59.132774 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.132751    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/475195a2-42a8-493d-ab68-585d187a977d-istiod-ca-cert\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:30:59.133958 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.133919    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-istio-envoy\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:30:59.134081 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.134055    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istio-envoy\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:30:59.134081 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.134067    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istio-podinfo\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:30:59.134258 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.134239    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/475195a2-42a8-493d-ab68-585d187a977d-istio-podinfo\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:30:59.156410 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.156372    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8g6pr\" (UniqueName: \"kubernetes.io/projected/b06d985f-9a38-4ea8-a120-676d0c3de8aa-kube-api-access-8g6pr\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:30:59.156813 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.156789    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istio-token\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:30:59.156887 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.156821    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bpkf\" (UniqueName: \"kubernetes.io/projected/475195a2-42a8-493d-ab68-585d187a977d-kube-api-access-2bpkf\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:30:59.156887 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.156787    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/475195a2-42a8-493d-ab68-585d187a977d-istio-token\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:30:59.206697 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.206656    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:30:59.258887 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.256522    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:30:59.526242 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.526216    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"]
Oct 31 21:30:59.527176 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:30:59.527148    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb06d985f_9a38_4ea8_a120_676d0c3de8aa.slice/crio-7ae70903fc3dfaf53fad680e06fd8769513ea49b1cdcbc21d5c255a66682b41d WatchSource:0}: Error finding container 7ae70903fc3dfaf53fad680e06fd8769513ea49b1cdcbc21d5c255a66682b41d: Status 404 returned error can't find the container with id 7ae70903fc3dfaf53fad680e06fd8769513ea49b1cdcbc21d5c255a66682b41d
Oct 31 21:30:59.529614 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.529580    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:30:59.529701 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.529648    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:30:59.529701 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.529674    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:30:59.581964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.581922    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"]
Oct 31 21:30:59.582737 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:30:59.582703    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod475195a2_42a8_493d_ab68_585d187a977d.slice/crio-156e4be1befc4b2fe091268c50abf7afb4a9225969753a2d08ad5bffdd1fe2eb WatchSource:0}: Error finding container 156e4be1befc4b2fe091268c50abf7afb4a9225969753a2d08ad5bffdd1fe2eb: Status 404 returned error can't find the container with id 156e4be1befc4b2fe091268c50abf7afb4a9225969753a2d08ad5bffdd1fe2eb
Oct 31 21:30:59.585300 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.585272    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:30:59.585375 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.585332    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:30:59.585375 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:30:59.585361    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:31:00.137962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:00.129556    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:00.229963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:00.214648    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85" event={"ID":"475195a2-42a8-493d-ab68-585d187a977d","Type":"ContainerStarted","Data":"6ce9e89101e01a48e317553d74bcc248326d0601a67cc7885defba4e1fef4d36"}
Oct 31 21:31:00.229963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:00.214824    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85" event={"ID":"475195a2-42a8-493d-ab68-585d187a977d","Type":"ContainerStarted","Data":"156e4be1befc4b2fe091268c50abf7afb4a9225969753a2d08ad5bffdd1fe2eb"}
Oct 31 21:31:00.292997 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:00.284378    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 31 21:31:00.292997 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:00.284434    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85" podUID="475195a2-42a8-493d-ab68-585d187a977d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 31 21:31:00.292997 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:00.292546    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85" podStartSLOduration=2.292524941 podStartE2EDuration="2.292524941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:30:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:31:00.283294638 +0000 UTC m=+900.199604068" watchObservedRunningTime="2026-10-31 21:31:00.292524941 +0000 UTC m=+900.208834370"
Oct 31 21:31:00.295328 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:00.295296    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7" event={"ID":"b06d985f-9a38-4ea8-a120-676d0c3de8aa","Type":"ContainerStarted","Data":"47718e5e2043427be3b2865aab093df5294356bbbb85f14b0aa7551f379ea9f0"}
Oct 31 21:31:00.295554 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:00.295524    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7" event={"ID":"b06d985f-9a38-4ea8-a120-676d0c3de8aa","Type":"ContainerStarted","Data":"7ae70903fc3dfaf53fad680e06fd8769513ea49b1cdcbc21d5c255a66682b41d"}
Oct 31 21:31:00.295660 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:00.295569    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:31:00.382266 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:00.382217    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7" podStartSLOduration=2.382203506 podStartE2EDuration="2.382203506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:30:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:31:00.380637928 +0000 UTC m=+900.296947356" watchObservedRunningTime="2026-10-31 21:31:00.382203506 +0000 UTC m=+900.298512935"
Oct 31 21:31:00.387297 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:00.387271    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ssl9_26069598-5dc4-4c96-b55f-dd2f09884419/ovn-acl-logging/0.log"
Oct 31 21:31:00.387611 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:00.387556    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ssl9_26069598-5dc4-4c96-b55f-dd2f09884419/ovn-acl-logging/0.log"
Oct 31 21:31:01.208163 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:01.208119    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:31:01.208772 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:01.208748    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 31 21:31:01.208871 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:01.208804    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7" podUID="b06d985f-9a38-4ea8-a120-676d0c3de8aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 31 21:31:01.264583 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:01.264546    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 31 21:31:01.264856 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:01.264819    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85" podUID="475195a2-42a8-493d-ab68-585d187a977d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 31 21:31:02.072965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:02.072498    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:02.207023 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:02.206975    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 31 21:31:02.207222 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:02.207057    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7" podUID="b06d985f-9a38-4ea8-a120-676d0c3de8aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 31 21:31:02.257963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:02.257223    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 31 21:31:02.257963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:02.257290    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85" podUID="475195a2-42a8-493d-ab68-585d187a977d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 31 21:31:03.181510 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.178072    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"]
Oct 31 21:31:03.185968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.184324    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 21:31:03.207282 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.207255    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 31 21:31:03.207451 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.207307    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7" podUID="b06d985f-9a38-4ea8-a120-676d0c3de8aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 31 21:31:03.218617 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.218593    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"]
Oct 31 21:31:03.276834 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.276798    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 31 21:31:03.277274 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.277022    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85" podUID="475195a2-42a8-493d-ab68-585d187a977d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 31 21:31:03.318229 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.318202    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-istio-data\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 21:31:03.318420 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.318257    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-workload-certs\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 21:31:03.318420 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.318337    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e97b2260-7cb0-4f61-a862-25b9313beb18-istio-podinfo\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 21:31:03.318420 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.318413    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lq6h7\" (UniqueName: \"kubernetes.io/projected/e97b2260-7cb0-4f61-a862-25b9313beb18-kube-api-access-lq6h7\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 21:31:03.318593 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.318471    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e97b2260-7cb0-4f61-a862-25b9313beb18-istio-token\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 21:31:03.318593 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.318514    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-credential-socket\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 21:31:03.318593 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.318546    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-istio-envoy\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 21:31:03.318750 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.318592    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-workload-socket\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 21:31:03.318750 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.318617    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e97b2260-7cb0-4f61-a862-25b9313beb18-istiod-ca-cert\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 21:31:03.419720 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.419682    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lq6h7\" (UniqueName: \"kubernetes.io/projected/e97b2260-7cb0-4f61-a862-25b9313beb18-kube-api-access-lq6h7\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 21:31:03.419890 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.419731    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e97b2260-7cb0-4f61-a862-25b9313beb18-istio-token\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 21:31:03.419890 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.419757    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-credential-socket\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 21:31:03.419890 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.419775    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-istio-envoy\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 21:31:03.419890 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.419800    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-workload-socket\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 21:31:03.419890 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.419815    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e97b2260-7cb0-4f61-a862-25b9313beb18-istiod-ca-cert\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 21:31:03.419890 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.419864    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-istio-data\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 21:31:03.420221 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.419893    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-workload-certs\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 21:31:03.420221 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.419934    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e97b2260-7cb0-4f61-a862-25b9313beb18-istio-podinfo\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 21:31:03.434018 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.432597    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e97b2260-7cb0-4f61-a862-25b9313beb18-istio-podinfo\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 21:31:03.434018 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.433479    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-credential-socket\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 21:31:03.458420 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.454457    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-workload-socket\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 21:31:03.458420 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.455292    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e97b2260-7cb0-4f61-a862-25b9313beb18-istiod-ca-cert\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 21:31:03.458420 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.455322    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-workload-certs\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 21:31:03.458420 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.455613    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-istio-data\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 21:31:03.471973 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.463801    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-istio-envoy\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 21:31:03.474583 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.472912    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lq6h7\" (UniqueName: \"kubernetes.io/projected/e97b2260-7cb0-4f61-a862-25b9313beb18-kube-api-access-lq6h7\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 21:31:03.476976 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.476875    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e97b2260-7cb0-4f61-a862-25b9313beb18-istio-token\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 21:31:03.504517 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.504489    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 21:31:03.805518 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.805418    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:03.882929 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.882904    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"]
Oct 31 21:31:03.884485 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:31:03.884459    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode97b2260_7cb0_4f61_a862_25b9313beb18.slice/crio-9245a5af7a9295c78eb0b3362d36a75b74ac41ac265c22be051cfe0a35b9974c WatchSource:0}: Error finding container 9245a5af7a9295c78eb0b3362d36a75b74ac41ac265c22be051cfe0a35b9974c: Status 404 returned error can't find the container with id 9245a5af7a9295c78eb0b3362d36a75b74ac41ac265c22be051cfe0a35b9974c
Oct 31 21:31:03.887034 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.886999    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:31:03.887127 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.887065    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:31:03.887127 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.887093    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:31:03.975019 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:03.974507    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:04.210363 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:04.210272    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 31 21:31:04.210619 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:04.210586    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7" podUID="b06d985f-9a38-4ea8-a120-676d0c3de8aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 31 21:31:04.264964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:04.261196    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 31 21:31:04.264964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:04.261260    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85" podUID="475195a2-42a8-493d-ab68-585d187a977d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 31 21:31:04.310076 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:04.310038    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df" event={"ID":"e97b2260-7cb0-4f61-a862-25b9313beb18","Type":"ContainerStarted","Data":"6f9b7a3fa6046f4630e0994091add07ba46808e925339a939754db6e63472ffa"}
Oct 31 21:31:04.310460 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:04.310106    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df" event={"ID":"e97b2260-7cb0-4f61-a862-25b9313beb18","Type":"ContainerStarted","Data":"9245a5af7a9295c78eb0b3362d36a75b74ac41ac265c22be051cfe0a35b9974c"}
Oct 31 21:31:04.383954 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:04.383875    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df" podStartSLOduration=1.383853188 podStartE2EDuration="1.383853188s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:31:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:31:04.375679284 +0000 UTC m=+904.291988738" watchObservedRunningTime="2026-10-31 21:31:04.383853188 +0000 UTC m=+904.300162618"
Oct 31 21:31:04.505393 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:04.505299    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 21:31:04.506549 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:04.506508    4460 patch_prober.go:29] interesting pod/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 31 21:31:04.506713 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:04.506564    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df" podUID="e97b2260-7cb0-4f61-a862-25b9313beb18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 31 21:31:05.207119 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:05.207082    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 31 21:31:05.207273 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:05.207132    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7" podUID="b06d985f-9a38-4ea8-a120-676d0c3de8aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 31 21:31:05.262463 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:05.262426    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 31 21:31:05.262776 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:05.262731    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85" podUID="475195a2-42a8-493d-ab68-585d187a977d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 31 21:31:05.505208 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:05.505104    4460 patch_prober.go:29] interesting pod/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 31 21:31:05.505208 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:05.505194    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df" podUID="e97b2260-7cb0-4f61-a862-25b9313beb18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 31 21:31:05.979966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:05.974621    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:06.098035 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:06.097640    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:31:06.207784 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:06.207750    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Oct 31 21:31:06.207994 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:06.207803    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7" podUID="b06d985f-9a38-4ea8-a120-676d0c3de8aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Oct 31 21:31:06.257691 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:06.257601    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Oct 31 21:31:06.257691 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:06.257677    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85" podUID="475195a2-42a8-493d-ab68-585d187a977d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Oct 31 21:31:06.505701 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:06.505654    4460 patch_prober.go:29] interesting pod/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 31 21:31:06.506099 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:06.505722    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df" podUID="e97b2260-7cb0-4f61-a862-25b9313beb18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 31 21:31:07.285980 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:07.284793    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:31:07.287069 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:07.286638    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:31:07.312964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:07.302198    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:31:07.513959 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:07.503671    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:31:07.513959 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:07.505111    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:31:07.578054 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:07.516256    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:31:07.578054 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:07.516661    4460 patch_prober.go:29] interesting pod/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 31 21:31:07.578054 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:07.516710    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df" podUID="e97b2260-7cb0-4f61-a862-25b9313beb18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 31 21:31:07.767626 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:07.767530    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:07.955246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:07.941736    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:08.142707 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:08.142672    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-9888fbfd6-xfvf9"]
Oct 31 21:31:08.419967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:08.419587    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-9888fbfd6-xfvf9"
Oct 31 21:31:08.519689 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:08.519649    4460 patch_prober.go:29] interesting pod/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 31 21:31:08.542085 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:08.542034    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df" podUID="e97b2260-7cb0-4f61-a862-25b9313beb18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 31 21:31:08.948968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:08.948050    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:31:09.009564 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:09.009514    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" podUID="af7e0e0c-cf52-4270-b528-35718339996c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:31:09.511961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:09.511908    4460 patch_prober.go:29] interesting pod/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 31 21:31:09.512166 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:09.511982    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df" podUID="e97b2260-7cb0-4f61-a862-25b9313beb18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 31 21:31:09.935028 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:09.934179    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:10.505095 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:10.505062    4460 patch_prober.go:29] interesting pod/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 31 21:31:10.505244 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:10.505120    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df" podUID="e97b2260-7cb0-4f61-a862-25b9313beb18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 31 21:31:11.505266 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:11.505227    4460 patch_prober.go:29] interesting pod/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 31 21:31:11.505642 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:11.505307    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df" podUID="e97b2260-7cb0-4f61-a862-25b9313beb18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 31 21:31:12.740364 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:12.740322    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"]
Oct 31 21:31:12.745665 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:12.745641    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:31:12.786178 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:12.786139    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"]
Oct 31 21:31:12.788308 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:12.788279    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:12.810173 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:12.810137    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-istio-envoy\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:31:12.810323 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:12.810180    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d136f2ed-85d7-4e13-936a-032a993576f0-istio-token\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:31:12.810323 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:12.810216    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d136f2ed-85d7-4e13-936a-032a993576f0-istio-podinfo\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:31:12.810457 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:12.810362    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-credential-socket\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:31:12.810530 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:12.810500    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d136f2ed-85d7-4e13-936a-032a993576f0-istiod-ca-cert\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:31:12.810610 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:12.810592    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-istio-data\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:31:12.810669 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:12.810626    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-workload-certs\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:31:12.810723 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:12.810705    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-workload-socket\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:31:12.810789 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:12.810774    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbklc\" (UniqueName: \"kubernetes.io/projected/d136f2ed-85d7-4e13-936a-032a993576f0-kube-api-access-fbklc\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:31:12.911721 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:12.911674    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-credential-socket\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:31:12.911869 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:12.911780    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d136f2ed-85d7-4e13-936a-032a993576f0-istiod-ca-cert\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:31:12.911869 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:12.911829    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-istio-data\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:31:12.911869 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:12.911862    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-workload-certs\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:31:12.912038 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:12.911912    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-workload-socket\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:31:12.912038 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:12.911982    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbklc\" (UniqueName: \"kubernetes.io/projected/d136f2ed-85d7-4e13-936a-032a993576f0-kube-api-access-fbklc\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:31:12.912132 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:12.912106    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-istio-envoy\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:31:12.912168 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:12.912133    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d136f2ed-85d7-4e13-936a-032a993576f0-istio-token\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:31:12.912208 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:12.912181    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d136f2ed-85d7-4e13-936a-032a993576f0-istio-podinfo\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:31:12.912208 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:12.912183    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-credential-socket\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:31:12.912603 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:12.912583    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-workload-certs\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:31:12.912704 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:12.912621    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d136f2ed-85d7-4e13-936a-032a993576f0-istiod-ca-cert\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:31:12.912821 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:12.912802    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-workload-socket\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:31:12.913178 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:12.913123    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-istio-data\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:31:12.915330 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:12.915290    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d136f2ed-85d7-4e13-936a-032a993576f0-istio-podinfo\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:31:12.918169 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:12.918147    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-istio-envoy\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:31:12.939056 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:12.939029    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d136f2ed-85d7-4e13-936a-032a993576f0-istio-token\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:31:12.939531 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:12.939512    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbklc\" (UniqueName: \"kubernetes.io/projected/d136f2ed-85d7-4e13-936a-032a993576f0-kube-api-access-fbklc\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:31:13.065616 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.065581    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:31:13.149585 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.145018    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-9888fbfd6-xfvf9" podUID="ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79" containerName="payload-processing" containerID="cri-o://e17718ccb01af32c5a6298ef762e08380e20f15e0cfdce9abff95e1945fff81c" gracePeriod=25
Oct 31 21:31:13.287700 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.287666    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 21:31:13.288779 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.288738    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 21:31:13.290638 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.290579    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 21:31:13.395530 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.395490    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"]
Oct 31 21:31:13.397406 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:31:13.397379    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd136f2ed_85d7_4e13_936a_032a993576f0.slice/crio-0e4ef95bbcffd650bc967ed73f2afe2290206633417988c0df9631ffd4d05913 WatchSource:0}: Error finding container 0e4ef95bbcffd650bc967ed73f2afe2290206633417988c0df9631ffd4d05913: Status 404 returned error can't find the container with id 0e4ef95bbcffd650bc967ed73f2afe2290206633417988c0df9631ffd4d05913
Oct 31 21:31:13.401985 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.400627    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:31:13.401985 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.400726    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:31:13.401985 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.400788    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:31:13.418278 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.418253    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-9888fbfd6-xfvf9"
Oct 31 21:31:13.515118 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.515078    4460 generic.go:356] "Generic (PLEG): container finished" podID="ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79" containerID="e17718ccb01af32c5a6298ef762e08380e20f15e0cfdce9abff95e1945fff81c" exitCode=0
Oct 31 21:31:13.515118 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.515121    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-9888fbfd6-xfvf9"
Oct 31 21:31:13.515353 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.515162    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-9888fbfd6-xfvf9" event={"ID":"ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79","Type":"ContainerDied","Data":"e17718ccb01af32c5a6298ef762e08380e20f15e0cfdce9abff95e1945fff81c"}
Oct 31 21:31:13.515353 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.515244    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-9888fbfd6-xfvf9" event={"ID":"ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79","Type":"ContainerDied","Data":"a872d687c7b13648b656bc5a76ed51bd16387dc15c3b2123ea71faf12335cfcf"}
Oct 31 21:31:13.515353 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.515277    4460 scope.go:122] "RemoveContainer" containerID="e17718ccb01af32c5a6298ef762e08380e20f15e0cfdce9abff95e1945fff81c"
Oct 31 21:31:13.518045 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.517995    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk" event={"ID":"d136f2ed-85d7-4e13-936a-032a993576f0","Type":"ContainerStarted","Data":"0e8f88b4ca2b74283874bc2e479d26b0c84850bc3bce64596a4941d95956fbbc"}
Oct 31 21:31:13.518227 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.518203    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk" event={"ID":"d136f2ed-85d7-4e13-936a-032a993576f0","Type":"ContainerStarted","Data":"0e4ef95bbcffd650bc967ed73f2afe2290206633417988c0df9631ffd4d05913"}
Oct 31 21:31:13.534664 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.524832    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79-plugins-config-volume\") pod \"ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79\" (UID: \"ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79\") "
Oct 31 21:31:13.534664 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.524992    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79-provider-credentials\" (UniqueName: \"kubernetes.io/projected/ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79-provider-credentials\") pod \"ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79\" (UID: \"ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79\") "
Oct 31 21:31:13.534664 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.525040    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79-routing-overlay-volume\") pod \"ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79\" (UID: \"ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79\") "
Oct 31 21:31:13.534664 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.525145    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79-kube-api-access-7llpm\" (UniqueName: \"kubernetes.io/projected/ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79-kube-api-access-7llpm\") pod \"ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79\" (UID: \"ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79\") "
Oct 31 21:31:13.534664 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.531025    4460 scope.go:122] "RemoveContainer" containerID="e17718ccb01af32c5a6298ef762e08380e20f15e0cfdce9abff95e1945fff81c"
Oct 31 21:31:13.534664 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.531182    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79-kube-api-access-7llpm" pod "ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79" (UID: "ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79"). InnerVolumeSpecName "kube-api-access-7llpm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:31:13.534664 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:31:13.531444    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e17718ccb01af32c5a6298ef762e08380e20f15e0cfdce9abff95e1945fff81c\": container with ID starting with e17718ccb01af32c5a6298ef762e08380e20f15e0cfdce9abff95e1945fff81c not found: ID does not exist" containerID="e17718ccb01af32c5a6298ef762e08380e20f15e0cfdce9abff95e1945fff81c"
Oct 31 21:31:13.534664 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.531493    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e17718ccb01af32c5a6298ef762e08380e20f15e0cfdce9abff95e1945fff81c"} err="failed to get container status \"e17718ccb01af32c5a6298ef762e08380e20f15e0cfdce9abff95e1945fff81c\": rpc error: code = NotFound desc = could not find container \"e17718ccb01af32c5a6298ef762e08380e20f15e0cfdce9abff95e1945fff81c\": container with ID starting with e17718ccb01af32c5a6298ef762e08380e20f15e0cfdce9abff95e1945fff81c not found: ID does not exist"
Oct 31 21:31:13.534664 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.531558    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79-plugins-config-volume" pod "ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79" (UID: "ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:31:13.534664 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.531844    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79-routing-overlay-volume" pod "ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79" (UID: "ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:31:13.534664 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.532740    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79-provider-credentials" pod "ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79" (UID: "ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:31:13.580166 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.580081    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk" podStartSLOduration=1.580067331 podStartE2EDuration="1.580067331s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:31:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:31:13.579630482 +0000 UTC m=+913.495939935" watchObservedRunningTime="2026-10-31 21:31:13.580067331 +0000 UTC m=+913.496376760"
Oct 31 21:31:13.627346 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.627315    4460 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79-routing-overlay-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:31:13.627346 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.627340    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7llpm\" (UniqueName: \"kubernetes.io/projected/ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79-kube-api-access-7llpm\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:31:13.627346 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.627349    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:31:13.627346 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.627358    4460 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79-provider-credentials\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:31:13.866212 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.866106    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-9888fbfd6-xfvf9"]
Oct 31 21:31:13.869048 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:13.869026    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-9888fbfd6-xfvf9"]
Oct 31 21:31:14.065859 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:14.065822    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:31:14.067282 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:14.067257    4460 patch_prober.go:29] interesting pod/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 31 21:31:14.067371 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:14.067316    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk" podUID="d136f2ed-85d7-4e13-936a-032a993576f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 31 21:31:14.249874 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:14.249792    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79" path="/var/lib/kubelet/pods/ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79/volumes"
Oct 31 21:31:14.741627 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:14.741458    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:15.066636 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:15.066601    4460 patch_prober.go:29] interesting pod/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 31 21:31:15.067032 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:15.066666    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk" podUID="d136f2ed-85d7-4e13-936a-032a993576f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 31 21:31:16.066776 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:16.066747    4460 patch_prober.go:29] interesting pod/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 31 21:31:16.067173 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:16.066807    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk" podUID="d136f2ed-85d7-4e13-936a-032a993576f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 31 21:31:16.613555 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:16.613516    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:17.066880 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:17.066833    4460 patch_prober.go:29] interesting pod/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 31 21:31:17.067327 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:17.066911    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk" podUID="d136f2ed-85d7-4e13-936a-032a993576f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 31 21:31:17.297119 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:17.296787    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"]
Oct 31 21:31:17.297428 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:17.297410    4460 cpu_manager.go:414] "containerMap: removing container" podUID="ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79" containerName="payload-processing"
Oct 31 21:31:17.297428 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:17.297427    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79" containerName="payload-processing"
Oct 31 21:31:17.297564 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:17.297514    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="ae4c7e5e-3405-46aa-9f7f-faaef8a4cd79" containerName="payload-processing"
Oct 31 21:31:17.302154 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:17.302122    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:31:17.347655 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:17.347573    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-994336\"/\"kube-root-ca.crt\""
Oct 31 21:31:17.350033 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:17.348065    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-994336\"/\"openshift-service-ca.crt\""
Oct 31 21:31:17.350033 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:17.348180    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-994336\"/\"distinct-w1-994336-kserve-self-signed-certs\""
Oct 31 21:31:17.350183 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:17.350071    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-994336\"/\"default-dockercfg-dp8bn\""
Oct 31 21:31:17.351830 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:17.351806    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"]
Oct 31 21:31:17.357570 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:17.357545    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:17.388733 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:17.388706    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-tls-certs\") pod \"distinct-w1-994336-kserve-84c46f5f8-jf747\" (UID: \"d1078aa7-1947-45ce-b6bb-d6550ad61fcf\") " pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:31:17.388839 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:17.388764    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-tmp-dir\") pod \"distinct-w1-994336-kserve-84c46f5f8-jf747\" (UID: \"d1078aa7-1947-45ce-b6bb-d6550ad61fcf\") " pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:31:17.388906 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:17.388848    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-model-cache\") pod \"distinct-w1-994336-kserve-84c46f5f8-jf747\" (UID: \"d1078aa7-1947-45ce-b6bb-d6550ad61fcf\") " pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:31:17.388998 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:17.388978    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwzsf\" (UniqueName: \"kubernetes.io/projected/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-kube-api-access-fwzsf\") pod \"distinct-w1-994336-kserve-84c46f5f8-jf747\" (UID: \"d1078aa7-1947-45ce-b6bb-d6550ad61fcf\") " pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:31:17.389048 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:17.389013    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-home\") pod \"distinct-w1-994336-kserve-84c46f5f8-jf747\" (UID: \"d1078aa7-1947-45ce-b6bb-d6550ad61fcf\") " pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:31:17.389108 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:17.389096    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-dshm\") pod \"distinct-w1-994336-kserve-84c46f5f8-jf747\" (UID: \"d1078aa7-1947-45ce-b6bb-d6550ad61fcf\") " pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:31:17.496620 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:17.496578    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-tls-certs\") pod \"distinct-w1-994336-kserve-84c46f5f8-jf747\" (UID: \"d1078aa7-1947-45ce-b6bb-d6550ad61fcf\") " pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:31:17.496888 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:17.496869    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-tmp-dir\") pod \"distinct-w1-994336-kserve-84c46f5f8-jf747\" (UID: \"d1078aa7-1947-45ce-b6bb-d6550ad61fcf\") " pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:31:17.497054 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:17.497036    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-model-cache\") pod \"distinct-w1-994336-kserve-84c46f5f8-jf747\" (UID: \"d1078aa7-1947-45ce-b6bb-d6550ad61fcf\") " pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:31:17.497221 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:17.497201    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwzsf\" (UniqueName: \"kubernetes.io/projected/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-kube-api-access-fwzsf\") pod \"distinct-w1-994336-kserve-84c46f5f8-jf747\" (UID: \"d1078aa7-1947-45ce-b6bb-d6550ad61fcf\") " pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:31:17.497335 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:17.497319    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-home\") pod \"distinct-w1-994336-kserve-84c46f5f8-jf747\" (UID: \"d1078aa7-1947-45ce-b6bb-d6550ad61fcf\") " pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:31:17.497426 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:17.497407    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-tmp-dir\") pod \"distinct-w1-994336-kserve-84c46f5f8-jf747\" (UID: \"d1078aa7-1947-45ce-b6bb-d6550ad61fcf\") " pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:31:17.497588 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:17.497569    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-dshm\") pod \"distinct-w1-994336-kserve-84c46f5f8-jf747\" (UID: \"d1078aa7-1947-45ce-b6bb-d6550ad61fcf\") " pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:31:17.497692 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:17.497673    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-model-cache\") pod \"distinct-w1-994336-kserve-84c46f5f8-jf747\" (UID: \"d1078aa7-1947-45ce-b6bb-d6550ad61fcf\") " pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:31:17.506966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:17.498508    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-home\") pod \"distinct-w1-994336-kserve-84c46f5f8-jf747\" (UID: \"d1078aa7-1947-45ce-b6bb-d6550ad61fcf\") " pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:31:17.507508 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:17.507483    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-dshm\") pod \"distinct-w1-994336-kserve-84c46f5f8-jf747\" (UID: \"d1078aa7-1947-45ce-b6bb-d6550ad61fcf\") " pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:31:17.533604 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:17.533566    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-tls-certs\") pod \"distinct-w1-994336-kserve-84c46f5f8-jf747\" (UID: \"d1078aa7-1947-45ce-b6bb-d6550ad61fcf\") " pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:31:17.542075 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:17.534123    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwzsf\" (UniqueName: \"kubernetes.io/projected/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-kube-api-access-fwzsf\") pod \"distinct-w1-994336-kserve-84c46f5f8-jf747\" (UID: \"d1078aa7-1947-45ce-b6bb-d6550ad61fcf\") " pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:31:17.616240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:17.616153    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:31:17.965280 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:17.965250    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"]
Oct 31 21:31:17.966638 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:31:17.966614    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd1078aa7_1947_45ce_b6bb_d6550ad61fcf.slice/crio-bb20163a65ccda8ed79cbcffbb395a10dd64666b5ba1ef922fb8d9841017c158 WatchSource:0}: Error finding container bb20163a65ccda8ed79cbcffbb395a10dd64666b5ba1ef922fb8d9841017c158: Status 404 returned error can't find the container with id bb20163a65ccda8ed79cbcffbb395a10dd64666b5ba1ef922fb8d9841017c158
Oct 31 21:31:18.067323 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:18.066830    4460 patch_prober.go:29] interesting pod/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 31 21:31:18.067323 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:18.066873    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk" podUID="d136f2ed-85d7-4e13-936a-032a993576f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 31 21:31:18.566147 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:18.566108    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:18.588596 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:18.588556    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747" event={"ID":"d1078aa7-1947-45ce-b6bb-d6550ad61fcf","Type":"ContainerStarted","Data":"af5b231520ddabdca0df6d92f164fe8c6fc8afa26d8ec336367e967e5f5d8350"}
Oct 31 21:31:18.588776 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:18.588601    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747" event={"ID":"d1078aa7-1947-45ce-b6bb-d6550ad61fcf","Type":"ContainerStarted","Data":"bb20163a65ccda8ed79cbcffbb395a10dd64666b5ba1ef922fb8d9841017c158"}
Oct 31 21:31:18.623996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:18.623965    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:31:18.664635 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:18.664564    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747" podStartSLOduration=1.664528996 podStartE2EDuration="1.664528996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:31:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:31:18.658186935 +0000 UTC m=+918.574496360" watchObservedRunningTime="2026-10-31 21:31:18.664528996 +0000 UTC m=+918.580838427"
Oct 31 21:31:19.066062 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:19.066027    4460 patch_prober.go:29] interesting pod/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 31 21:31:19.066222 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:19.066083    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk" podUID="d136f2ed-85d7-4e13-936a-032a993576f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 31 21:31:20.070211 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:20.070181    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:31:20.070619 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:20.070584    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:31:20.071219 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:20.071201    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:31:20.336246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:20.336162    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:21.451991 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:31:21.451954    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:31:22.020330 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:22.020293    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:23.716261 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:23.716224    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:26.892155 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:26.892095    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:31:27.161020 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:27.159070    4460 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podUID="9cd67bf5-8833-4a82-bc33-bed7bdfd4d36" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:31:27.622026 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:27.621990    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:31:27.671966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:27.671047    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podUID="9cd67bf5-8833-4a82-bc33-bed7bdfd4d36" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": context deadline exceeded"
Oct 31 21:31:27.897147 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:27.897034    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:31:27.898889 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:27.898867    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:31:27.964987 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:27.961025    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" podUID="af7e0e0c-cf52-4270-b528-35718339996c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Oct 31 21:31:28.089779 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:28.089491    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz" podUID="93a42128-ee74-4dcd-b158-27e33cfd44f9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded"
Oct 31 21:31:28.674962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:28.674370    4460 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747" podUID="d1078aa7-1947-45ce-b6bb-d6550ad61fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/health\": context deadline exceeded"
Oct 31 21:31:28.736966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:28.701299    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podUID="9cd67bf5-8833-4a82-bc33-bed7bdfd4d36" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:31:28.736966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:28.701786    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:31:28.908399 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:28.908265    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:31:28.961330 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:28.961196    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" podUID="af7e0e0c-cf52-4270-b528-35718339996c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:31:28.962046 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:28.962009    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:31:29.003059 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:29.003026    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:31:29.120988 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:29.120928    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:29.183079 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:29.183037    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz" podUID="93a42128-ee74-4dcd-b158-27e33cfd44f9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:31:29.214015 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:29.191484    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:31:29.538231 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:29.538203    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:31:29.558631 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:29.558600    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:31:29.588224 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:29.588197    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:31:31.633618 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:31.633580    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:32.419597 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:32.419564    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:34.105710 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:34.105650    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:35.731195 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:35.731158    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:37.435195 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:37.435158    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:37.629798 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:37.629759    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:31:37.640779 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:37.640750    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:31:39.134159 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:39.134124    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:40.873561 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:40.873523    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:42.540639 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:42.540597    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:43.272259 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:43.272217    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:31:44.267474 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:44.267438    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:45.053151 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:45.048818    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:46.694698 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:46.694650    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:47.025223 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:47.025185    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"]
Oct 31 21:31:47.030893 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:47.030863    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:31:47.052441 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:47.052366    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"]
Oct 31 21:31:47.061180 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:47.061146    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-106555\"/\"kube-root-ca.crt\""
Oct 31 21:31:47.061345 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:47.061219    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-106555\"/\"default-dockercfg-g8p4t\""
Oct 31 21:31:47.061345 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:47.061311    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-106555\"/\"openshift-service-ca.crt\""
Oct 31 21:31:47.061345 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:47.061315    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-106555\"/\"distinct-2-w3-106555-kserve-self-signed-certs\""
Oct 31 21:31:47.096828 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:47.096797    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:47.139427 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:47.139386    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9771611f-879a-44f6-9c1a-f8647dc2d841-model-cache\") pod \"distinct-2-w3-106555-kserve-78f45b6fc5-m44xt\" (UID: \"9771611f-879a-44f6-9c1a-f8647dc2d841\") " pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:31:47.139595 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:47.139523    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpnfb\" (UniqueName: \"kubernetes.io/projected/9771611f-879a-44f6-9c1a-f8647dc2d841-kube-api-access-vpnfb\") pod \"distinct-2-w3-106555-kserve-78f45b6fc5-m44xt\" (UID: \"9771611f-879a-44f6-9c1a-f8647dc2d841\") " pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:31:47.139595 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:47.139548    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9771611f-879a-44f6-9c1a-f8647dc2d841-dshm\") pod \"distinct-2-w3-106555-kserve-78f45b6fc5-m44xt\" (UID: \"9771611f-879a-44f6-9c1a-f8647dc2d841\") " pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:31:47.139595 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:47.139568    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9771611f-879a-44f6-9c1a-f8647dc2d841-tls-certs\") pod \"distinct-2-w3-106555-kserve-78f45b6fc5-m44xt\" (UID: \"9771611f-879a-44f6-9c1a-f8647dc2d841\") " pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:31:47.139595 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:47.139593    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9771611f-879a-44f6-9c1a-f8647dc2d841-tmp-dir\") pod \"distinct-2-w3-106555-kserve-78f45b6fc5-m44xt\" (UID: \"9771611f-879a-44f6-9c1a-f8647dc2d841\") " pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:31:47.139744 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:47.139649    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9771611f-879a-44f6-9c1a-f8647dc2d841-home\") pod \"distinct-2-w3-106555-kserve-78f45b6fc5-m44xt\" (UID: \"9771611f-879a-44f6-9c1a-f8647dc2d841\") " pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:31:47.249961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:47.249774    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9771611f-879a-44f6-9c1a-f8647dc2d841-model-cache\") pod \"distinct-2-w3-106555-kserve-78f45b6fc5-m44xt\" (UID: \"9771611f-879a-44f6-9c1a-f8647dc2d841\") " pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:31:47.250149 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:47.250014    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpnfb\" (UniqueName: \"kubernetes.io/projected/9771611f-879a-44f6-9c1a-f8647dc2d841-kube-api-access-vpnfb\") pod \"distinct-2-w3-106555-kserve-78f45b6fc5-m44xt\" (UID: \"9771611f-879a-44f6-9c1a-f8647dc2d841\") " pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:31:47.250149 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:47.250064    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9771611f-879a-44f6-9c1a-f8647dc2d841-dshm\") pod \"distinct-2-w3-106555-kserve-78f45b6fc5-m44xt\" (UID: \"9771611f-879a-44f6-9c1a-f8647dc2d841\") " pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:31:47.250149 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:47.250100    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9771611f-879a-44f6-9c1a-f8647dc2d841-tls-certs\") pod \"distinct-2-w3-106555-kserve-78f45b6fc5-m44xt\" (UID: \"9771611f-879a-44f6-9c1a-f8647dc2d841\") " pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:31:47.250149 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:47.250141    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9771611f-879a-44f6-9c1a-f8647dc2d841-tmp-dir\") pod \"distinct-2-w3-106555-kserve-78f45b6fc5-m44xt\" (UID: \"9771611f-879a-44f6-9c1a-f8647dc2d841\") " pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:31:47.250372 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:47.250167    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9771611f-879a-44f6-9c1a-f8647dc2d841-home\") pod \"distinct-2-w3-106555-kserve-78f45b6fc5-m44xt\" (UID: \"9771611f-879a-44f6-9c1a-f8647dc2d841\") " pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:31:47.262995 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:47.250890    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9771611f-879a-44f6-9c1a-f8647dc2d841-home\") pod \"distinct-2-w3-106555-kserve-78f45b6fc5-m44xt\" (UID: \"9771611f-879a-44f6-9c1a-f8647dc2d841\") " pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:31:47.262995 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:47.251220    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9771611f-879a-44f6-9c1a-f8647dc2d841-model-cache\") pod \"distinct-2-w3-106555-kserve-78f45b6fc5-m44xt\" (UID: \"9771611f-879a-44f6-9c1a-f8647dc2d841\") " pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:31:47.262995 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:47.256283    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9771611f-879a-44f6-9c1a-f8647dc2d841-tmp-dir\") pod \"distinct-2-w3-106555-kserve-78f45b6fc5-m44xt\" (UID: \"9771611f-879a-44f6-9c1a-f8647dc2d841\") " pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:31:47.276211 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:47.273626    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9771611f-879a-44f6-9c1a-f8647dc2d841-tls-certs\") pod \"distinct-2-w3-106555-kserve-78f45b6fc5-m44xt\" (UID: \"9771611f-879a-44f6-9c1a-f8647dc2d841\") " pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:31:47.293487 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:47.290216    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9771611f-879a-44f6-9c1a-f8647dc2d841-dshm\") pod \"distinct-2-w3-106555-kserve-78f45b6fc5-m44xt\" (UID: \"9771611f-879a-44f6-9c1a-f8647dc2d841\") " pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:31:47.293487 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:47.290886    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpnfb\" (UniqueName: \"kubernetes.io/projected/9771611f-879a-44f6-9c1a-f8647dc2d841-kube-api-access-vpnfb\") pod \"distinct-2-w3-106555-kserve-78f45b6fc5-m44xt\" (UID: \"9771611f-879a-44f6-9c1a-f8647dc2d841\") " pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:31:47.347446 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:47.347410    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:31:47.652184 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:47.652158    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"]
Oct 31 21:31:47.654480 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:31:47.654446    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9771611f_879a_44f6_9c1a_f8647dc2d841.slice/crio-187ea0133245631e2eba9644ff0c9cb82636271953b240038587b85968fa472e WatchSource:0}: Error finding container 187ea0133245631e2eba9644ff0c9cb82636271953b240038587b85968fa472e: Status 404 returned error can't find the container with id 187ea0133245631e2eba9644ff0c9cb82636271953b240038587b85968fa472e
Oct 31 21:31:48.060444 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.060393    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt" event={"ID":"9771611f-879a-44f6-9c1a-f8647dc2d841","Type":"ContainerStarted","Data":"73d9aaf9d7c0c2fe045ecf2b1a443e5efdee1811083a06bc560d6c8256490b9c"}
Oct 31 21:31:48.060444 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.060450    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt" event={"ID":"9771611f-879a-44f6-9c1a-f8647dc2d841","Type":"ContainerStarted","Data":"187ea0133245631e2eba9644ff0c9cb82636271953b240038587b85968fa472e"}
Oct 31 21:31:48.107401 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.107347    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt" podStartSLOduration=2.10733281 podStartE2EDuration="2.10733281s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:31:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:31:48.106018049 +0000 UTC m=+948.022327481" watchObservedRunningTime="2026-10-31 21:31:48.10733281 +0000 UTC m=+948.023642239"
Oct 31 21:31:48.313876 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.313797    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"]
Oct 31 21:31:48.317991 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.317966    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:31:48.345786 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.345749    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"]
Oct 31 21:31:48.348307 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.348276    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:31:48.381320 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.381279    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/849e440a-7e18-4e25-90b3-bdc437076436-istio-podinfo\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:31:48.381503 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.381335    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-credential-socket\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:31:48.381503 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.381421    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwkwj\" (UniqueName: \"kubernetes.io/projected/849e440a-7e18-4e25-90b3-bdc437076436-kube-api-access-dwkwj\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:31:48.381503 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.381475    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-workload-certs\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:31:48.382202 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.381704    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/849e440a-7e18-4e25-90b3-bdc437076436-istiod-ca-cert\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:31:48.382202 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.381852    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-istio-data\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:31:48.382202 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.381893    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/849e440a-7e18-4e25-90b3-bdc437076436-istio-token\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:31:48.382202 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.381927    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-istio-envoy\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:31:48.382202 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.382084    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-workload-socket\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:31:48.483732 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.483696    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-credential-socket\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:31:48.483732 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.483747    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwkwj\" (UniqueName: \"kubernetes.io/projected/849e440a-7e18-4e25-90b3-bdc437076436-kube-api-access-dwkwj\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:31:48.483997 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.483789    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-workload-certs\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:31:48.483997 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.483865    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/849e440a-7e18-4e25-90b3-bdc437076436-istiod-ca-cert\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:31:48.483997 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.483921    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-istio-data\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:31:48.483997 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.483972    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/849e440a-7e18-4e25-90b3-bdc437076436-istio-token\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:31:48.484198 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.484003    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-istio-envoy\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:31:48.484198 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.484035    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-workload-socket\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:31:48.484198 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.484173    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/849e440a-7e18-4e25-90b3-bdc437076436-istio-podinfo\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:31:48.484198 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.484183    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-credential-socket\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:31:48.484393 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.484236    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-workload-certs\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:31:48.484393 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.484308    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-istio-data\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:31:48.484594 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.484569    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-workload-socket\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:31:48.484663 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.484567    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/849e440a-7e18-4e25-90b3-bdc437076436-istiod-ca-cert\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:31:48.486436 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.486414    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-istio-envoy\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:31:48.486703 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.486686    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/849e440a-7e18-4e25-90b3-bdc437076436-istio-podinfo\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:31:48.494249 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.494220    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:48.509059 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.509040    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwkwj\" (UniqueName: \"kubernetes.io/projected/849e440a-7e18-4e25-90b3-bdc437076436-kube-api-access-dwkwj\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:31:48.509138 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.509061    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/849e440a-7e18-4e25-90b3-bdc437076436-istio-token\") pod \"e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") " pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:31:48.634370 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.634279    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:31:48.960568 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.960540    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"]
Oct 31 21:31:48.961257 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:31:48.961236    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod849e440a_7e18_4e25_90b3_bdc437076436.slice/crio-c6ff6dfbb050f02129e97bb86c981a06165f1403281e7020590e25cf246c9c75 WatchSource:0}: Error finding container c6ff6dfbb050f02129e97bb86c981a06165f1403281e7020590e25cf246c9c75: Status 404 returned error can't find the container with id c6ff6dfbb050f02129e97bb86c981a06165f1403281e7020590e25cf246c9c75
Oct 31 21:31:48.963651 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.963625    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:31:48.963730 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.963680    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:31:48.963730 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:48.963708    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:31:49.069792 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:49.069753    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv" event={"ID":"849e440a-7e18-4e25-90b3-bdc437076436","Type":"ContainerStarted","Data":"55f94ef8a50b7ba4e526e91f01ee45b433a397dc0739c946f1ad46bda2ee7458"}
Oct 31 21:31:49.070257 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:49.069829    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv" event={"ID":"849e440a-7e18-4e25-90b3-bdc437076436","Type":"ContainerStarted","Data":"c6ff6dfbb050f02129e97bb86c981a06165f1403281e7020590e25cf246c9c75"}
Oct 31 21:31:49.121131 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:49.121080    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv" podStartSLOduration=1.1210556 podStartE2EDuration="1.1210556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:31:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:31:49.119007611 +0000 UTC m=+949.035317041" watchObservedRunningTime="2026-10-31 21:31:49.1210556 +0000 UTC m=+949.037365028"
Oct 31 21:31:49.635022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:49.634982    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:31:50.644966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:50.641931    4460 patch_prober.go:29] interesting pod/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 31 21:31:50.644966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:50.642016    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv" podUID="849e440a-7e18-4e25-90b3-bdc437076436" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 31 21:31:50.764302 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:50.764251    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:51.469613 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:31:51.469574    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:31:51.650144 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:51.650106    4460 patch_prober.go:29] interesting pod/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 31 21:31:51.650707 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:51.650674    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv" podUID="849e440a-7e18-4e25-90b3-bdc437076436" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 31 21:31:52.642080 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:52.642037    4460 patch_prober.go:29] interesting pod/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 31 21:31:52.642378 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:52.642349    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv" podUID="849e440a-7e18-4e25-90b3-bdc437076436" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 31 21:31:52.744965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:52.741448    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:53.636986 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:53.636379    4460 patch_prober.go:29] interesting pod/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 31 21:31:53.636986 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:53.636445    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv" podUID="849e440a-7e18-4e25-90b3-bdc437076436" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 31 21:31:54.638010 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:54.637972    4460 patch_prober.go:29] interesting pod/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 31 21:31:54.638613 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:54.638570    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv" podUID="849e440a-7e18-4e25-90b3-bdc437076436" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 31 21:31:55.077073 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:55.077034    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:55.635421 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:55.635386    4460 patch_prober.go:29] interesting pod/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 31 21:31:55.635620 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:55.635438    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv" podUID="849e440a-7e18-4e25-90b3-bdc437076436" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 31 21:31:56.639866 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:56.639833    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:31:56.640364 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:56.640213    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:31:56.641003 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:56.640979    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:31:57.113154 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:57.113119    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:31:57.348422 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:57.348385    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:31:57.360907 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:57.360879    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:31:58.135322 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:58.135294    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:31:58.958772 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:31:58.958723    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:00.741216 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:00.741131    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:02.513658 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:02.513620    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:04.482959 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:04.482902    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:05.593891 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:05.593845    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podUID="9cd67bf5-8833-4a82-bc33-bed7bdfd4d36" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:05.637693 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:05.636712    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747" podUID="d1078aa7-1947-45ce-b6bb-d6550ad61fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 31 21:32:06.001026 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:05.976016    4460 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:06.039713 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:06.039669    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" podUID="af7e0e0c-cf52-4270-b528-35718339996c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:06.146434 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:06.146357    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt" podUID="9771611f-879a-44f6-9c1a-f8647dc2d841" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:06.731967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:06.729884    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz" podUID="93a42128-ee74-4dcd-b158-27e33cfd44f9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:06.837253 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:06.837205    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747" podUID="d1078aa7-1947-45ce-b6bb-d6550ad61fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:06.847392 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:06.846029    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podUID="9cd67bf5-8833-4a82-bc33-bed7bdfd4d36" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:06.847392 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:06.846850    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:06.902914 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:06.893684    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:32:06.903277 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:06.903248    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:32:07.254966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:07.236559    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt" podUID="9771611f-879a-44f6-9c1a-f8647dc2d841" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:07.452994 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:07.259447    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:32:07.453353 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:07.290455    4460 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podUID="9cd67bf5-8833-4a82-bc33-bed7bdfd4d36" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:07.460961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:07.342316    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:07.465127 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:07.343381    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" podUID="af7e0e0c-cf52-4270-b528-35718339996c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Oct 31 21:32:07.478220 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:07.478189    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:32:07.500966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:07.497546    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:32:07.691974 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:07.683296    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:07.731554 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:07.731492    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz" podUID="93a42128-ee74-4dcd-b158-27e33cfd44f9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded"
Oct 31 21:32:07.739028 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:07.739000    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:32:07.764105 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:07.764075    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:32:07.772420 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:07.772389    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:32:07.860966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:07.860009    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 31 21:32:07.860966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:07.860425    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podUID="9cd67bf5-8833-4a82-bc33-bed7bdfd4d36" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": context deadline exceeded"
Oct 31 21:32:07.860966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:07.860959    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:32:07.874012 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:07.873983    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:32:07.902578 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:07.902414    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:32:07.906197 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:07.906172    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:32:08.303871 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:08.303832    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"]
Oct 31 21:32:08.310035 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:08.310007    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:32:08.326676 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:08.326648    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-cc45a7\"/\"distinct-2-w4-cc45a7-kserve-self-signed-certs\""
Oct 31 21:32:08.329433 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:08.329407    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"]
Oct 31 21:32:08.365295 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:08.365249    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:08.381291 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:08.381261    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5zp9\" (UniqueName: \"kubernetes.io/projected/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-kube-api-access-v5zp9\") pod \"distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n\" (UID: \"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf\") " pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:32:08.381444 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:08.381369    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-tls-certs\") pod \"distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n\" (UID: \"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf\") " pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:32:08.381487 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:08.381459    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-tmp-dir\") pod \"distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n\" (UID: \"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf\") " pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:32:08.381560 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:08.381548    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-model-cache\") pod \"distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n\" (UID: \"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf\") " pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:32:08.381615 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:08.381595    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-home\") pod \"distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n\" (UID: \"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf\") " pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:32:08.381667 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:08.381615    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-dshm\") pod \"distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n\" (UID: \"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf\") " pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:32:08.498964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:08.494419    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5zp9\" (UniqueName: \"kubernetes.io/projected/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-kube-api-access-v5zp9\") pod \"distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n\" (UID: \"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf\") " pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:32:08.498964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:08.494584    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-tls-certs\") pod \"distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n\" (UID: \"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf\") " pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:32:08.498964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:08.494705    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-tmp-dir\") pod \"distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n\" (UID: \"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf\") " pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:32:08.498964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:08.494796    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-model-cache\") pod \"distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n\" (UID: \"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf\") " pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:32:08.498964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:08.494863    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-home\") pod \"distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n\" (UID: \"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf\") " pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:32:08.498964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:08.494886    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-dshm\") pod \"distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n\" (UID: \"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf\") " pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:32:08.523962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:08.518447    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-model-cache\") pod \"distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n\" (UID: \"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf\") " pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:32:08.523962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:08.519015    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-home\") pod \"distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n\" (UID: \"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf\") " pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:32:08.523962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:08.521839    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-dshm\") pod \"distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n\" (UID: \"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf\") " pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:32:08.523962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:08.522231    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-tmp-dir\") pod \"distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n\" (UID: \"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf\") " pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:32:08.557308 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:08.556618    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-tls-certs\") pod \"distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n\" (UID: \"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf\") " pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:32:08.583962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:08.583665    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5zp9\" (UniqueName: \"kubernetes.io/projected/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-kube-api-access-v5zp9\") pod \"distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n\" (UID: \"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf\") " pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:32:08.634590 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:08.633891    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:32:08.940983 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:08.940957    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"]
Oct 31 21:32:08.943108 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:32:08.942955    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4cf6bb6a_3a46_41f0_9e4f_f0fabbbc82cf.slice/crio-0856fa69738f548a68a5add24df6d887d2936f87523d84dd53a8f3d8ca944e27 WatchSource:0}: Error finding container 0856fa69738f548a68a5add24df6d887d2936f87523d84dd53a8f3d8ca944e27: Status 404 returned error can't find the container with id 0856fa69738f548a68a5add24df6d887d2936f87523d84dd53a8f3d8ca944e27
Oct 31 21:32:09.762081 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:09.762039    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n" event={"ID":"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf","Type":"ContainerStarted","Data":"2240fefaacf7242c77b2590ed5769617ea2f979ba0174268e69c0c6661e5f5c0"}
Oct 31 21:32:09.762081 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:09.762083    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n" event={"ID":"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf","Type":"ContainerStarted","Data":"0856fa69738f548a68a5add24df6d887d2936f87523d84dd53a8f3d8ca944e27"}
Oct 31 21:32:09.812057 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:09.812006    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n" podStartSLOduration=1.811991355 podStartE2EDuration="1.811991355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:32:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:32:09.811155912 +0000 UTC m=+969.727465343" watchObservedRunningTime="2026-10-31 21:32:09.811991355 +0000 UTC m=+969.728300785"
Oct 31 21:32:10.517206 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:10.517163    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:10.641080 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:10.638673    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:32:12.322024 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:12.321982    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:12.759792 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:12.759766    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"]
Oct 31 21:32:12.772479 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:12.772445    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 21:32:12.784787 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:12.784743    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"]
Oct 31 21:32:12.791725 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:12.791697    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-994336\"/\"distinct-2-w1-994336-kserve-self-signed-certs\""
Oct 31 21:32:12.809728 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:12.809690    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:12.888663 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:12.888629    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/869ed893-352c-4f8f-a174-b42afdeb4de7-home\") pod \"distinct-2-w1-994336-kserve-d779f66fc-z5pgh\" (UID: \"869ed893-352c-4f8f-a174-b42afdeb4de7\") " pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 21:32:12.888848 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:12.888679    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/869ed893-352c-4f8f-a174-b42afdeb4de7-model-cache\") pod \"distinct-2-w1-994336-kserve-d779f66fc-z5pgh\" (UID: \"869ed893-352c-4f8f-a174-b42afdeb4de7\") " pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 21:32:12.888848 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:12.888775    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/869ed893-352c-4f8f-a174-b42afdeb4de7-tmp-dir\") pod \"distinct-2-w1-994336-kserve-d779f66fc-z5pgh\" (UID: \"869ed893-352c-4f8f-a174-b42afdeb4de7\") " pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 21:32:12.888848 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:12.888833    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4lf7\" (UniqueName: \"kubernetes.io/projected/869ed893-352c-4f8f-a174-b42afdeb4de7-kube-api-access-v4lf7\") pod \"distinct-2-w1-994336-kserve-d779f66fc-z5pgh\" (UID: \"869ed893-352c-4f8f-a174-b42afdeb4de7\") " pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 21:32:12.889053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:12.889023    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/869ed893-352c-4f8f-a174-b42afdeb4de7-dshm\") pod \"distinct-2-w1-994336-kserve-d779f66fc-z5pgh\" (UID: \"869ed893-352c-4f8f-a174-b42afdeb4de7\") " pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 21:32:12.889130 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:12.889116    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/869ed893-352c-4f8f-a174-b42afdeb4de7-tls-certs\") pod \"distinct-2-w1-994336-kserve-d779f66fc-z5pgh\" (UID: \"869ed893-352c-4f8f-a174-b42afdeb4de7\") " pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 21:32:13.005486 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:13.005445    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/869ed893-352c-4f8f-a174-b42afdeb4de7-tmp-dir\") pod \"distinct-2-w1-994336-kserve-d779f66fc-z5pgh\" (UID: \"869ed893-352c-4f8f-a174-b42afdeb4de7\") " pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 21:32:13.006339 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:13.006314    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4lf7\" (UniqueName: \"kubernetes.io/projected/869ed893-352c-4f8f-a174-b42afdeb4de7-kube-api-access-v4lf7\") pod \"distinct-2-w1-994336-kserve-d779f66fc-z5pgh\" (UID: \"869ed893-352c-4f8f-a174-b42afdeb4de7\") " pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 21:32:13.006963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:13.006925    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/869ed893-352c-4f8f-a174-b42afdeb4de7-dshm\") pod \"distinct-2-w1-994336-kserve-d779f66fc-z5pgh\" (UID: \"869ed893-352c-4f8f-a174-b42afdeb4de7\") " pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 21:32:13.007537 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:13.007517    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/869ed893-352c-4f8f-a174-b42afdeb4de7-tls-certs\") pod \"distinct-2-w1-994336-kserve-d779f66fc-z5pgh\" (UID: \"869ed893-352c-4f8f-a174-b42afdeb4de7\") " pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 21:32:13.018049 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:13.017964    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/869ed893-352c-4f8f-a174-b42afdeb4de7-home\") pod \"distinct-2-w1-994336-kserve-d779f66fc-z5pgh\" (UID: \"869ed893-352c-4f8f-a174-b42afdeb4de7\") " pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 21:32:13.018686 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:13.018663    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/869ed893-352c-4f8f-a174-b42afdeb4de7-model-cache\") pod \"distinct-2-w1-994336-kserve-d779f66fc-z5pgh\" (UID: \"869ed893-352c-4f8f-a174-b42afdeb4de7\") " pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 21:32:13.019303 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:13.018585    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/869ed893-352c-4f8f-a174-b42afdeb4de7-home\") pod \"distinct-2-w1-994336-kserve-d779f66fc-z5pgh\" (UID: \"869ed893-352c-4f8f-a174-b42afdeb4de7\") " pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 21:32:13.019447 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:13.006214    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/869ed893-352c-4f8f-a174-b42afdeb4de7-tmp-dir\") pod \"distinct-2-w1-994336-kserve-d779f66fc-z5pgh\" (UID: \"869ed893-352c-4f8f-a174-b42afdeb4de7\") " pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 21:32:13.028087 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:13.019109    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/869ed893-352c-4f8f-a174-b42afdeb4de7-model-cache\") pod \"distinct-2-w1-994336-kserve-d779f66fc-z5pgh\" (UID: \"869ed893-352c-4f8f-a174-b42afdeb4de7\") " pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 21:32:13.053974 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:13.030044    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/869ed893-352c-4f8f-a174-b42afdeb4de7-tls-certs\") pod \"distinct-2-w1-994336-kserve-d779f66fc-z5pgh\" (UID: \"869ed893-352c-4f8f-a174-b42afdeb4de7\") " pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 21:32:13.054668 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:13.054639    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/869ed893-352c-4f8f-a174-b42afdeb4de7-dshm\") pod \"distinct-2-w1-994336-kserve-d779f66fc-z5pgh\" (UID: \"869ed893-352c-4f8f-a174-b42afdeb4de7\") " pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 21:32:13.062643 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:13.062608    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4lf7\" (UniqueName: \"kubernetes.io/projected/869ed893-352c-4f8f-a174-b42afdeb4de7-kube-api-access-v4lf7\") pod \"distinct-2-w1-994336-kserve-d779f66fc-z5pgh\" (UID: \"869ed893-352c-4f8f-a174-b42afdeb4de7\") " pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 21:32:13.102229 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:13.102197    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 21:32:13.423172 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:13.423140    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"]
Oct 31 21:32:13.424174 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:32:13.424147    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod869ed893_352c_4f8f_a174_b42afdeb4de7.slice/crio-c13e9df596a4709caa644e1ca02b5810a028ca74112dd30a82c1e2dc4bf9ae53 WatchSource:0}: Error finding container c13e9df596a4709caa644e1ca02b5810a028ca74112dd30a82c1e2dc4bf9ae53: Status 404 returned error can't find the container with id c13e9df596a4709caa644e1ca02b5810a028ca74112dd30a82c1e2dc4bf9ae53
Oct 31 21:32:13.794279 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:13.794227    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh" event={"ID":"869ed893-352c-4f8f-a174-b42afdeb4de7","Type":"ContainerStarted","Data":"e0817b2f9f16f4aec1b829e0d1daf6425a927c079717e4a035f854e4b64b4b46"}
Oct 31 21:32:13.794465 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:13.794288    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh" event={"ID":"869ed893-352c-4f8f-a174-b42afdeb4de7","Type":"ContainerStarted","Data":"c13e9df596a4709caa644e1ca02b5810a028ca74112dd30a82c1e2dc4bf9ae53"}
Oct 31 21:32:13.840395 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:13.840334    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh" podStartSLOduration=1.840316345 podStartE2EDuration="1.840316345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:32:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:32:13.838653285 +0000 UTC m=+973.754962717" watchObservedRunningTime="2026-10-31 21:32:13.840316345 +0000 UTC m=+973.756625778"
Oct 31 21:32:14.041208 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:14.041166    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:14.104854 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:14.104709    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 21:32:15.732480 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:15.732442    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:17.380696 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:17.380659    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:18.634841 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:18.634799    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:32:18.648276 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:18.648238    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:32:18.859138 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:18.859104    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:32:19.108849 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:19.108814    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:20.743821 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:20.743782    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:20.964304 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:20.964264    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:21.084484 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:21.084439    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:21.209229 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:21.209192    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:21.454983 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:32:21.454881    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:32:21.792810 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:21.792776    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:32:22.150090 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:22.150006    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:22.271104 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:22.271073    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:22.385996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:22.385959    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:22.653074 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:22.653038    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:23.106765 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:23.106730    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 21:32:23.302577 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:23.302546    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 21:32:24.779965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:24.779229    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz" podUID="93a42128-ee74-4dcd-b158-27e33cfd44f9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:24.832964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:24.832106    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747" podUID="d1078aa7-1947-45ce-b6bb-d6550ad61fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:24.864466 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:24.864043    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podUID="9cd67bf5-8833-4a82-bc33-bed7bdfd4d36" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:24.887969 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:24.876063    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" podUID="af7e0e0c-cf52-4270-b528-35718339996c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:24.913025 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:24.906104    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 31 21:32:24.937136 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:24.937066    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh" podUID="869ed893-352c-4f8f-a174-b42afdeb4de7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:25.141724 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:25.129041    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt" podUID="9771611f-879a-44f6-9c1a-f8647dc2d841" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 31 21:32:25.784166 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:25.784092    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz" podUID="93a42128-ee74-4dcd-b158-27e33cfd44f9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:25.786492 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:25.785628    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:32:26.031982 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:26.006212    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747" podUID="d1078aa7-1947-45ce-b6bb-d6550ad61fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 31 21:32:26.031982 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:26.006561    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n" podUID="4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:26.031982 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:26.006829    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podUID="9cd67bf5-8833-4a82-bc33-bed7bdfd4d36" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:26.031982 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:26.007107    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" podUID="af7e0e0c-cf52-4270-b528-35718339996c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Oct 31 21:32:26.031982 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:26.016821    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:32:26.031982 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:26.016897    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:32:26.031982 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:26.016959    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:32:26.031982 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:26.019320    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh" podUID="869ed893-352c-4f8f-a174-b42afdeb4de7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.77:8000/ready\": context deadline exceeded"
Oct 31 21:32:26.066106 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:26.063385    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:26.066106 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:26.063815    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:32:26.188089 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:26.188037    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt" podUID="9771611f-879a-44f6-9c1a-f8647dc2d841" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:26.191765 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:26.191729    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:32:26.800959 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:26.273308    4460 patch_prober.go:29] interesting pod/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 21:32:26.800959 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:26.799902    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz" podUID="93a42128-ee74-4dcd-b158-27e33cfd44f9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:26.833233 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:26.273703    4460 patch_prober.go:29] interesting pod/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 21:32:26.833233 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:26.831044    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc" podUID="9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:26.833233 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:26.435068    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 21:32:26.833233 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:26.831983    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9" podUID="9501cf33-637a-4d00-8e58-3ae157dd6840" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:26.850077 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:26.850029    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z" podUID="ea04c855-12eb-4628-a947-3b4e8321bef1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:27.013814 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:27.011293    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" podUID="af7e0e0c-cf52-4270-b528-35718339996c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:27.035033 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:27.028533    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n" podUID="4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:27.035033 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:27.028894    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podUID="9cd67bf5-8833-4a82-bc33-bed7bdfd4d36" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:27.035033 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:27.029220    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747" podUID="d1078aa7-1947-45ce-b6bb-d6550ad61fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:27.087871 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:27.041276    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:32:27.087871 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:27.046428    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh" podUID="869ed893-352c-4f8f-a174-b42afdeb4de7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:27.087871 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:27.087039    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:27.172095 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:27.157541    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:32:27.273222 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:27.271774    4460 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podUID="9cd67bf5-8833-4a82-bc33-bed7bdfd4d36" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/health\": context deadline exceeded"
Oct 31 21:32:27.296993 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:27.290078    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:32:27.316513 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:27.316480    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:32:27.425578 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:27.425493    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 21:32:27.425840 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:27.425814    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk" podUID="3f60d565-50f1-4b9d-959a-014be6e49128" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:27.462539 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:27.462498    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 21:32:27.499592 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:27.498600    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:32:27.500272 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:27.500248    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:32:27.759363 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:27.759275    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:32:27.776392 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:27.764528    4460 patch_prober.go:29] interesting pod/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 21:32:27.776392 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:27.764589    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv" podUID="849e440a-7e18-4e25-90b3-bdc437076436" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:27.791230 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:27.791200    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:32:30.088413 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:30.088375    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:32.171462 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:32.171416    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:33.916646 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:33.916606    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:34.610656 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:34.610621    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:35.642559 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:35.642521    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:37.308096 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:37.308060    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:38.924380 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:38.924342    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:40.715092 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:40.715054    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:40.831975 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:40.829863    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:42.300551 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:42.300515    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:43.880840 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:43.880805    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:45.294409 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:45.294374    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:46.763968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:46.762036    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 31 21:32:46.780653 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:46.780610    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:46.940128 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:46.940079    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh" podUID="869ed893-352c-4f8f-a174-b42afdeb4de7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:47.039140 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:47.039036    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podUID="9cd67bf5-8833-4a82-bc33-bed7bdfd4d36" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:47.039779 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:47.039384    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747" podUID="d1078aa7-1947-45ce-b6bb-d6550ad61fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:47.150053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:47.150009    4460 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podUID="9cd67bf5-8833-4a82-bc33-bed7bdfd4d36" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:47.179106 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:47.179049    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt" podUID="9771611f-879a-44f6-9c1a-f8647dc2d841" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:47.288074 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:47.288027    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" podUID="af7e0e0c-cf52-4270-b528-35718339996c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:47.332240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:47.332122    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz" podUID="93a42128-ee74-4dcd-b158-27e33cfd44f9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:47.504965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:47.504026    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n" podUID="4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:47.778964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:47.764137    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:47.778964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:47.769088    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:32:47.944967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:47.942051    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh" podUID="869ed893-352c-4f8f-a174-b42afdeb4de7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.77:8000/ready\": context deadline exceeded"
Oct 31 21:32:47.944967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:47.942868    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 21:32:48.043034 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:48.042903    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747" podUID="d1078aa7-1947-45ce-b6bb-d6550ad61fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:48.050964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:48.043408    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podUID="9cd67bf5-8833-4a82-bc33-bed7bdfd4d36" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": context deadline exceeded"
Oct 31 21:32:48.050964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:48.043839    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:32:48.062953 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:48.062897    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:32:48.296511 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:48.296411    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" podUID="af7e0e0c-cf52-4270-b528-35718339996c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:48.334130 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:48.334084    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz" podUID="93a42128-ee74-4dcd-b158-27e33cfd44f9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:48.337536 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:48.337507    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:32:48.338562 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:48.338539    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:32:48.513964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:48.504302    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n" podUID="4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:48.513964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:48.507697    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:32:48.683172 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:48.683074    4460 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747" podUID="d1078aa7-1947-45ce-b6bb-d6550ad61fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/health\": context deadline exceeded"
Oct 31 21:32:48.764345 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:48.764299    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 31 21:32:48.954136 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:48.954032    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh" podUID="869ed893-352c-4f8f-a174-b42afdeb4de7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:49.044134 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:49.043130    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747" podUID="d1078aa7-1947-45ce-b6bb-d6550ad61fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 31 21:32:49.054928 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:49.054887    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podUID="9cd67bf5-8833-4a82-bc33-bed7bdfd4d36" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": context deadline exceeded"
Oct 31 21:32:49.212830 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:49.179390    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt" podUID="9771611f-879a-44f6-9c1a-f8647dc2d841" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:49.331139 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:49.301523    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" podUID="af7e0e0c-cf52-4270-b528-35718339996c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:49.473015 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:49.425440    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz" podUID="93a42128-ee74-4dcd-b158-27e33cfd44f9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded"
Oct 31 21:32:49.521131 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:49.521082    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n" podUID="4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:49.713080 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:49.713037    4460 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n" podUID="4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:32:49.839923 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:49.839291    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 31 21:32:49.953607 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:49.953570    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"]
Oct 31 21:32:49.968132 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:49.968104    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"]
Oct 31 21:32:49.968670 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:49.968653    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:32:49.968794 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:49.968784    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:32:49.968901 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:49.968892    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:32:49.971131 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:49.969907    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:32:49.973036 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:49.973013    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:32:49.998961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:49.994669    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 21:32:49.998961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:49.995558    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:32:50.002961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.002611    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:32:50.144976 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.144547    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istio-token\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:32:50.144976 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.144614    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istiod-ca-cert\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:32:50.144976 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.144675    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istio-podinfo\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:32:50.144976 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.144819    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-credential-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:32:50.145435 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.145002    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmp2v\" (UniqueName: \"kubernetes.io/projected/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-kube-api-access-dmp2v\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:32:50.145435 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.145177    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istio-envoy\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:32:50.145435 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.145246    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-workload-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:32:50.145435 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.145325    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istio-data\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:32:50.145435 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.145356    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-workload-certs\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:32:50.246964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.246806    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istio-envoy\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:32:50.246964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.246878    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-workload-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:32:50.246964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.246958    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istio-data\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:32:50.247259 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.246989    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-workload-certs\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:32:50.247259 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.247029    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istio-token\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:32:50.247259 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.247068    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istiod-ca-cert\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:32:50.247259 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.247111    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istio-podinfo\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:32:50.247259 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.247200    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-credential-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:32:50.247546 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.247304    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmp2v\" (UniqueName: \"kubernetes.io/projected/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-kube-api-access-dmp2v\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:32:50.276519 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.248681    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-workload-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:32:50.276519 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.248979    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istio-data\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:32:50.276519 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.249248    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-workload-certs\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:32:50.276519 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.251306    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-credential-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:32:50.276519 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.255114    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istiod-ca-cert\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:32:50.276519 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.266601    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istio-podinfo\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:32:50.280045 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.277643    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istio-envoy\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:32:50.296013 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.285622    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmp2v\" (UniqueName: \"kubernetes.io/projected/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-kube-api-access-dmp2v\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:32:50.297811 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.297779    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istio-token\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:32:50.429024 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.427298    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:32:50.792605 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:32:50.792577    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b5ffb34_9d39_4ec9_b4f8_5ed2d90d6daa.slice/crio-d45878df72b1da34423dd1922998cd9ad9d4713a33e3ccc813ad78907891ea28 WatchSource:0}: Error finding container d45878df72b1da34423dd1922998cd9ad9d4713a33e3ccc813ad78907891ea28: Status 404 returned error can't find the container with id d45878df72b1da34423dd1922998cd9ad9d4713a33e3ccc813ad78907891ea28
Oct 31 21:32:50.792973 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.792929    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"]
Oct 31 21:32:50.794999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.794971    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:32:50.795106 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.795043    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:32:50.795106 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.795083    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:32:50.906761 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.905600    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz" event={"ID":"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa","Type":"ContainerStarted","Data":"90521185611ba3d14449a6f5008199c6b2753b54d2ad2d45b7f638308d41bd4c"}
Oct 31 21:32:50.906761 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.905693    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz" event={"ID":"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa","Type":"ContainerStarted","Data":"d45878df72b1da34423dd1922998cd9ad9d4713a33e3ccc813ad78907891ea28"}
Oct 31 21:32:50.959268 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:50.959213    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz" podStartSLOduration=1.9591965 podStartE2EDuration="1.9591965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:32:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:32:50.956574734 +0000 UTC m=+1010.872884166" watchObservedRunningTime="2026-10-31 21:32:50.9591965 +0000 UTC m=+1010.875505930"
Oct 31 21:32:51.437034 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:51.434005    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:32:51.448972 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:51.448917    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 31 21:32:51.449137 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:51.448989    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz" podUID="2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 31 21:32:51.471034 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:32:51.470934    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:32:51.773389 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:51.773353    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:52.444969 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:52.430296    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 31 21:32:52.444969 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:52.430363    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz" podUID="2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 31 21:32:53.445097 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.438360    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 31 21:32:53.445097 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.438423    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz" podUID="2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 31 21:32:53.784911 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.784870    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"]
Oct 31 21:32:53.789332 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.789300    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:32:53.813846 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.813803    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"]
Oct 31 21:32:53.848476 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.848423    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71d27085-7e15-413d-a20c-9e81d3a1123a-istio-token\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:32:53.858606 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.858467    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"]
Oct 31 21:32:53.858843 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.858593    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-istio-data\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:32:53.858995 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.858877    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96h56\" (UniqueName: \"kubernetes.io/projected/71d27085-7e15-413d-a20c-9e81d3a1123a-kube-api-access-96h56\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:32:53.859141 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.859101    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-credential-socket\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:32:53.859293 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.859226    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-istio-envoy\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:32:53.859438 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.859418    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-workload-socket\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:32:53.859526 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.859467    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71d27085-7e15-413d-a20c-9e81d3a1123a-istio-podinfo\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:32:53.859526 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.859506    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-workload-certs\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:32:53.859670 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.859536    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71d27085-7e15-413d-a20c-9e81d3a1123a-istiod-ca-cert\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:32:53.864008 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.863981    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:32:53.869612 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.869579    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"]
Oct 31 21:32:53.960583 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.960545    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-credential-socket\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:32:53.960773 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.960608    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-istio-envoy\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:32:53.960773 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.960629    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-istio-data\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:32:53.960773 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.960648    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/692f0926-1719-44a1-94a9-12604a64cf40-istio-podinfo\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:32:53.960773 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.960665    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/692f0926-1719-44a1-94a9-12604a64cf40-istio-token\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:32:53.961041 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.960882    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4hrh\" (UniqueName: \"kubernetes.io/projected/692f0926-1719-44a1-94a9-12604a64cf40-kube-api-access-q4hrh\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:32:53.961041 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.960931    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-credential-socket\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:32:53.961041 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.960935    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-credential-socket\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:32:53.961186 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.961051    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-workload-socket\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:32:53.961186 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.961080    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71d27085-7e15-413d-a20c-9e81d3a1123a-istio-podinfo\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:32:53.961186 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.961099    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-workload-socket\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:32:53.961350 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.961215    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-workload-certs\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:32:53.961350 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.961311    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71d27085-7e15-413d-a20c-9e81d3a1123a-istiod-ca-cert\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:32:53.961350 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.961345    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-workload-certs\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:32:53.961504 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.961402    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-istio-envoy\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:32:53.961504 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.961403    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-workload-socket\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:32:53.961504 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.961485    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-workload-certs\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:32:53.961611 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.961555    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71d27085-7e15-413d-a20c-9e81d3a1123a-istio-token\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:32:53.961652 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.961615    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-istio-data\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:32:53.961652 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.961639    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96h56\" (UniqueName: \"kubernetes.io/projected/71d27085-7e15-413d-a20c-9e81d3a1123a-kube-api-access-96h56\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:32:53.961744 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.961667    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/692f0926-1719-44a1-94a9-12604a64cf40-istiod-ca-cert\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:32:53.961996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.961973    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-istio-data\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:32:53.962104 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.962033    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71d27085-7e15-413d-a20c-9e81d3a1123a-istiod-ca-cert\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:32:53.963033 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.963009    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-istio-envoy\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:32:53.963376 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.963359    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71d27085-7e15-413d-a20c-9e81d3a1123a-istio-podinfo\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:32:53.987743 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.987715    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71d27085-7e15-413d-a20c-9e81d3a1123a-istio-token\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:32:53.987891 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:53.987844    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96h56\" (UniqueName: \"kubernetes.io/projected/71d27085-7e15-413d-a20c-9e81d3a1123a-kube-api-access-96h56\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:32:54.063539 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.063453    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4hrh\" (UniqueName: \"kubernetes.io/projected/692f0926-1719-44a1-94a9-12604a64cf40-kube-api-access-q4hrh\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:32:54.063539 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.063502    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-credential-socket\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:32:54.063732 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.063561    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-workload-socket\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:32:54.063732 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.063698    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-workload-certs\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:32:54.063820 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.063758    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-istio-envoy\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:32:54.063866 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.063852    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/692f0926-1719-44a1-94a9-12604a64cf40-istiod-ca-cert\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:32:54.064173 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.063981    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-istio-data\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:32:54.064173 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.064016    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/692f0926-1719-44a1-94a9-12604a64cf40-istio-podinfo\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:32:54.064173 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.064016    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-credential-socket\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:32:54.064173 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.064031    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/692f0926-1719-44a1-94a9-12604a64cf40-istio-token\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:32:54.064173 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.064126    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-workload-socket\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:32:54.064497 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.064368    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-istio-data\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:32:54.064497 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.064376    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-workload-certs\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:32:54.064746 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.064723    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/692f0926-1719-44a1-94a9-12604a64cf40-istiod-ca-cert\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:32:54.066291 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.066268    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-istio-envoy\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:32:54.066452 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.066432    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/692f0926-1719-44a1-94a9-12604a64cf40-istio-podinfo\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:32:54.097497 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.097467    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/692f0926-1719-44a1-94a9-12604a64cf40-istio-token\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:32:54.097622 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.097560    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4hrh\" (UniqueName: \"kubernetes.io/projected/692f0926-1719-44a1-94a9-12604a64cf40-kube-api-access-q4hrh\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:32:54.106491 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.106466    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:32:54.181100 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.181062    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:32:54.302847 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.302272    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:54.483226 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.483138    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 31 21:32:54.483817 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.483775    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz" podUID="2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 31 21:32:54.496698 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.496663    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"]
Oct 31 21:32:54.498597 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:32:54.498562    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71d27085_7e15_413d_a20c_9e81d3a1123a.slice/crio-384c63b7f6c4d4e4e1e57b9939cb578b8cde96daaa569e6413032d10975cccd6 WatchSource:0}: Error finding container 384c63b7f6c4d4e4e1e57b9939cb578b8cde96daaa569e6413032d10975cccd6: Status 404 returned error can't find the container with id 384c63b7f6c4d4e4e1e57b9939cb578b8cde96daaa569e6413032d10975cccd6
Oct 31 21:32:54.501331 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.501289    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:32:54.501445 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.501385    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:32:54.501445 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.501429    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:32:54.564689 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.563774    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"]
Oct 31 21:32:54.567077 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:32:54.567051    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod692f0926_1719_44a1_94a9_12604a64cf40.slice/crio-74448adf34340444036ac98b1ea48a59f0f765984347060429584887d4d6dec1 WatchSource:0}: Error finding container 74448adf34340444036ac98b1ea48a59f0f765984347060429584887d4d6dec1: Status 404 returned error can't find the container with id 74448adf34340444036ac98b1ea48a59f0f765984347060429584887d4d6dec1
Oct 31 21:32:54.570797 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.570762    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:32:54.570880 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.570830    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:32:54.570880 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.570869    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:32:54.667759 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.667714    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:54.963038 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.962998    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn" event={"ID":"71d27085-7e15-413d-a20c-9e81d3a1123a","Type":"ContainerStarted","Data":"b9bed6215de842a8743f88fccf2aa0bd29f0509ca08abfbf35070caf0169a83d"}
Oct 31 21:32:54.963709 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.963684    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn" event={"ID":"71d27085-7e15-413d-a20c-9e81d3a1123a","Type":"ContainerStarted","Data":"384c63b7f6c4d4e4e1e57b9939cb578b8cde96daaa569e6413032d10975cccd6"}
Oct 31 21:32:54.965674 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.965581    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh" event={"ID":"692f0926-1719-44a1-94a9-12604a64cf40","Type":"ContainerStarted","Data":"e22db340d79a599478fbd492503655a4a93f4c28ea4b2e7f1ab81155af3c21e4"}
Oct 31 21:32:54.965674 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:54.965638    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh" event={"ID":"692f0926-1719-44a1-94a9-12604a64cf40","Type":"ContainerStarted","Data":"74448adf34340444036ac98b1ea48a59f0f765984347060429584887d4d6dec1"}
Oct 31 21:32:55.017962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:55.017736    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn" podStartSLOduration=2.017714475 podStartE2EDuration="2.017714475s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:32:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:32:55.013317307 +0000 UTC m=+1014.929626738" watchObservedRunningTime="2026-10-31 21:32:55.017714475 +0000 UTC m=+1014.934023905"
Oct 31 21:32:55.067143 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:55.067074    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh" podStartSLOduration=2.067052832 podStartE2EDuration="2.067052832s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:32:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:32:55.06655308 +0000 UTC m=+1014.982862506" watchObservedRunningTime="2026-10-31 21:32:55.067052832 +0000 UTC m=+1014.983362263"
Oct 31 21:32:55.107551 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:55.107161    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:32:55.111491 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:55.111449    4460 patch_prober.go:29] interesting pod/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 31 21:32:55.111797 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:55.111522    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn" podUID="71d27085-7e15-413d-a20c-9e81d3a1123a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 31 21:32:55.183624 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:55.183584    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:32:55.183828 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:55.183712    4460 patch_prober.go:29] interesting pod/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 31 21:32:55.183828 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:55.183762    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh" podUID="692f0926-1719-44a1-94a9-12604a64cf40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 31 21:32:55.434962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:55.432385    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 31 21:32:55.434962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:55.432456    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz" podUID="2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 31 21:32:56.108068 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:56.108026    4460 patch_prober.go:29] interesting pod/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 31 21:32:56.108484 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:56.108084    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn" podUID="71d27085-7e15-413d-a20c-9e81d3a1123a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 31 21:32:56.186964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:56.186108    4460 patch_prober.go:29] interesting pod/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 31 21:32:56.186964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:56.186171    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh" podUID="692f0926-1719-44a1-94a9-12604a64cf40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 31 21:32:56.434450 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:56.434347    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 31 21:32:56.434738 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:56.434707    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz" podUID="2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 31 21:32:56.523966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:56.520474    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:57.107493 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:57.107452    4460 patch_prober.go:29] interesting pod/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 31 21:32:57.107675 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:57.107520    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn" podUID="71d27085-7e15-413d-a20c-9e81d3a1123a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 31 21:32:57.182439 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:57.182401    4460 patch_prober.go:29] interesting pod/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 31 21:32:57.182813 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:57.182452    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh" podUID="692f0926-1719-44a1-94a9-12604a64cf40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 31 21:32:57.434025 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:57.433421    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 31 21:32:57.434025 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:57.433488    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz" podUID="2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 31 21:32:58.107863 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:58.107825    4460 patch_prober.go:29] interesting pod/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 31 21:32:58.108068 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:58.107884    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn" podUID="71d27085-7e15-413d-a20c-9e81d3a1123a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 31 21:32:58.189417 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:58.189373    4460 patch_prober.go:29] interesting pod/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 31 21:32:58.189830 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:58.189438    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh" podUID="692f0926-1719-44a1-94a9-12604a64cf40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 31 21:32:58.596085 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:58.596028    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:32:58.888535 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:58.888457    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:32:58.889725 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:58.889695    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:32:58.891056 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:58.891034    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:32:59.110895 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:59.110857    4460 patch_prober.go:29] interesting pod/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 31 21:32:59.111511 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:59.111475    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn" podUID="71d27085-7e15-413d-a20c-9e81d3a1123a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 31 21:32:59.191250 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:59.191154    4460 patch_prober.go:29] interesting pod/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 31 21:32:59.191649 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:32:59.191268    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh" podUID="692f0926-1719-44a1-94a9-12604a64cf40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 31 21:33:00.107739 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:00.107700    4460 patch_prober.go:29] interesting pod/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 31 21:33:00.107929 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:00.107754    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn" podUID="71d27085-7e15-413d-a20c-9e81d3a1123a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 31 21:33:00.181879 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:00.181844    4460 patch_prober.go:29] interesting pod/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 31 21:33:00.182059 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:00.181901    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh" podUID="692f0926-1719-44a1-94a9-12604a64cf40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 31 21:33:00.676921 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:00.676748    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:01.107216 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:01.107175    4460 patch_prober.go:29] interesting pod/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 31 21:33:01.107525 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:01.107236    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn" podUID="71d27085-7e15-413d-a20c-9e81d3a1123a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 31 21:33:01.181696 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:01.181644    4460 patch_prober.go:29] interesting pod/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 31 21:33:01.181911 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:01.181731    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh" podUID="692f0926-1719-44a1-94a9-12604a64cf40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 31 21:33:02.188825 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:02.185345    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:33:02.188825 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:02.186663    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:33:02.188825 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:02.187288    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:33:02.300897 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:02.300837    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:33:02.302601 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:02.302569    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:33:02.307878 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:02.307853    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:33:02.361838 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:02.361805    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:04.012321 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:04.012280    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:05.844726 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:05.844689    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:07.565005 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:07.564959    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:08.070961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:08.070902    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:33:10.315967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:10.313606    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:10.470981 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:10.465973    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:10.682579 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:10.650385    4460 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz" podUID="93a42128-ee74-4dcd-b158-27e33cfd44f9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/health\": context deadline exceeded"
Oct 31 21:33:10.863960 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:10.863013    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:10.995044 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:10.993649    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:33:11.004064 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:11.004019    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh" podUID="869ed893-352c-4f8f-a174-b42afdeb4de7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.77:8000/ready\": context deadline exceeded"
Oct 31 21:33:11.004673 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:11.004634    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n" podUID="4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:33:11.030964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:11.007224    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz" podUID="93a42128-ee74-4dcd-b158-27e33cfd44f9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:33:11.113880 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:11.112471    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747" podUID="d1078aa7-1947-45ce-b6bb-d6550ad61fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:33:11.113880 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:11.113246    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podUID="9cd67bf5-8833-4a82-bc33-bed7bdfd4d36" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:33:11.162887 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:11.162752    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt" podUID="9771611f-879a-44f6-9c1a-f8647dc2d841" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:33:11.207760 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:11.207726    4460 patch_prober.go:29] interesting pod/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 21:33:11.208081 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:11.208037    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z" podUID="ea04c855-12eb-4628-a947-3b4e8321bef1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:33:11.367005 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:11.364838    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:11.411963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:11.390289    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" podUID="af7e0e0c-cf52-4270-b528-35718339996c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:33:11.996070 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:11.996021    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:33:11.997135 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:11.997111    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:33:12.007049 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:12.007015    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n" podUID="4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:33:12.007626 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:12.007597    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh" podUID="869ed893-352c-4f8f-a174-b42afdeb4de7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.77:8000/ready\": context deadline exceeded"
Oct 31 21:33:12.008989 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:12.008970    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:33:12.009221 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:12.009206    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 21:33:12.094963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:12.069579    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz" podUID="93a42128-ee74-4dcd-b158-27e33cfd44f9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:33:12.170659 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:12.170618    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:33:12.222764 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:12.222714    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt" podUID="9771611f-879a-44f6-9c1a-f8647dc2d841" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 31 21:33:12.249045 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:12.248453    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:33:12.300995 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:12.285023    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747" podUID="d1078aa7-1947-45ce-b6bb-d6550ad61fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 31 21:33:12.300995 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:12.285448    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podUID="9cd67bf5-8833-4a82-bc33-bed7bdfd4d36" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": context deadline exceeded"
Oct 31 21:33:12.377963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:12.366075    4460 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" podUID="af7e0e0c-cf52-4270-b528-35718339996c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:33:12.446687 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:12.446630    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" podUID="af7e0e0c-cf52-4270-b528-35718339996c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:33:12.705869 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:12.705838    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:33:12.706178 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:12.706159    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:33:12.706421 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:12.706403    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:33:13.007318 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:13.007068    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 31 21:33:13.007599 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:13.007567    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n" podUID="4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": context deadline exceeded"
Oct 31 21:33:13.008189 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:13.008159    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh" podUID="869ed893-352c-4f8f-a174-b42afdeb4de7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.77:8000/ready\": context deadline exceeded"
Oct 31 21:33:13.219573 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:13.219439    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz" podUID="93a42128-ee74-4dcd-b158-27e33cfd44f9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded"
Oct 31 21:33:13.236982 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:13.232768    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt" podUID="9771611f-879a-44f6-9c1a-f8647dc2d841" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:33:13.304965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:13.301657    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podUID="9cd67bf5-8833-4a82-bc33-bed7bdfd4d36" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": context deadline exceeded"
Oct 31 21:33:13.306868 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:13.305077    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747" podUID="d1078aa7-1947-45ce-b6bb-d6550ad61fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:33:13.350844 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:13.350241    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:33:13.466633 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:13.465254    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" podUID="af7e0e0c-cf52-4270-b528-35718339996c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Oct 31 21:33:13.477031 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:13.477001    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 21:33:13.737180 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:13.737093    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:33:13.913266 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:13.913221    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:33:13.918917 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:13.918884    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:33:13.934970 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:13.934273    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:33:13.934970 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:13.934375    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:33:13.998224 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:13.997988    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:33:14.597379 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:14.597339    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:15.395232 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:15.395180    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:15.844497 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:15.844461    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:16.446369 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:16.446326    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:16.965912 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:16.965874    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:18.838721 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:18.838685    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:19.188811 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:19.188714    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:19.301332 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:19.301289    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:20.163377 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:20.163250    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:20.285594 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:20.285564    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:20.741551 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:20.741514    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:20.905086 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:20.904839    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:21.031021 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:21.030981    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:21.106723 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:21.106687    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-90db52-57f7c9d8dc-ltfg6"]
Oct 31 21:33:21.111099 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:21.111078    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-90db52-57f7c9d8dc-ltfg6"
Oct 31 21:33:21.129122 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:21.129078    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-90db52\""
Oct 31 21:33:21.133895 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:21.133865    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-90db52-57f7c9d8dc-ltfg6"]
Oct 31 21:33:21.142263 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:21.142215    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-90db52-dockercfg-z5rvm\""
Oct 31 21:33:21.172564 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:21.172519    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-90db52-57478fddb7-vq79w"]
Oct 31 21:33:21.176816 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:21.176789    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-90db52-57478fddb7-vq79w"
Oct 31 21:33:21.246607 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:21.246553    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sh95j\" (UniqueName: \"kubernetes.io/projected/9afe6453-662b-4786-8f53-b9d6d720750a-kube-api-access-sh95j\") pod \"payload-pre-processing-e2e-api-a-90db52-57f7c9d8dc-ltfg6\" (UID: \"9afe6453-662b-4786-8f53-b9d6d720750a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-90db52-57f7c9d8dc-ltfg6"
Oct 31 21:33:21.247208 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:21.247178    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9afe6453-662b-4786-8f53-b9d6d720750a-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-90db52-57f7c9d8dc-ltfg6\" (UID: \"9afe6453-662b-4786-8f53-b9d6d720750a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-90db52-57f7c9d8dc-ltfg6"
Oct 31 21:33:21.356272 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:21.323383    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-90db52-57478fddb7-vq79w"]
Oct 31 21:33:21.455911 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:21.449904    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9afe6453-662b-4786-8f53-b9d6d720750a-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-90db52-57f7c9d8dc-ltfg6\" (UID: \"9afe6453-662b-4786-8f53-b9d6d720750a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-90db52-57f7c9d8dc-ltfg6"
Oct 31 21:33:21.455911 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:21.450016    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/086c327e-0e66-4f9c-83fb-85f688ffa253-plugins-config-volume\") pod \"payload-processing-e2e-api-a-90db52-57478fddb7-vq79w\" (UID: \"086c327e-0e66-4f9c-83fb-85f688ffa253\") " pod="openshift-ingress/payload-processing-e2e-api-a-90db52-57478fddb7-vq79w"
Oct 31 21:33:21.455911 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:21.450130    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wn2kh\" (UniqueName: \"kubernetes.io/projected/086c327e-0e66-4f9c-83fb-85f688ffa253-kube-api-access-wn2kh\") pod \"payload-processing-e2e-api-a-90db52-57478fddb7-vq79w\" (UID: \"086c327e-0e66-4f9c-83fb-85f688ffa253\") " pod="openshift-ingress/payload-processing-e2e-api-a-90db52-57478fddb7-vq79w"
Oct 31 21:33:21.455911 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:21.450293    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh95j\" (UniqueName: \"kubernetes.io/projected/9afe6453-662b-4786-8f53-b9d6d720750a-kube-api-access-sh95j\") pod \"payload-pre-processing-e2e-api-a-90db52-57f7c9d8dc-ltfg6\" (UID: \"9afe6453-662b-4786-8f53-b9d6d720750a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-90db52-57f7c9d8dc-ltfg6"
Oct 31 21:33:21.455911 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:21.453583    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9afe6453-662b-4786-8f53-b9d6d720750a-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-90db52-57f7c9d8dc-ltfg6\" (UID: \"9afe6453-662b-4786-8f53-b9d6d720750a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-90db52-57f7c9d8dc-ltfg6"
Oct 31 21:33:21.485981 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:21.482628    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sh95j\" (UniqueName: \"kubernetes.io/projected/9afe6453-662b-4786-8f53-b9d6d720750a-kube-api-access-sh95j\") pod \"payload-pre-processing-e2e-api-a-90db52-57f7c9d8dc-ltfg6\" (UID: \"9afe6453-662b-4786-8f53-b9d6d720750a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-90db52-57f7c9d8dc-ltfg6"
Oct 31 21:33:21.509562 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:33:21.509527    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:33:21.551848 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:21.551808    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/086c327e-0e66-4f9c-83fb-85f688ffa253-plugins-config-volume\") pod \"payload-processing-e2e-api-a-90db52-57478fddb7-vq79w\" (UID: \"086c327e-0e66-4f9c-83fb-85f688ffa253\") " pod="openshift-ingress/payload-processing-e2e-api-a-90db52-57478fddb7-vq79w"
Oct 31 21:33:21.552064 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:21.551896    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wn2kh\" (UniqueName: \"kubernetes.io/projected/086c327e-0e66-4f9c-83fb-85f688ffa253-kube-api-access-wn2kh\") pod \"payload-processing-e2e-api-a-90db52-57478fddb7-vq79w\" (UID: \"086c327e-0e66-4f9c-83fb-85f688ffa253\") " pod="openshift-ingress/payload-processing-e2e-api-a-90db52-57478fddb7-vq79w"
Oct 31 21:33:21.552666 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:21.552646    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/086c327e-0e66-4f9c-83fb-85f688ffa253-plugins-config-volume\") pod \"payload-processing-e2e-api-a-90db52-57478fddb7-vq79w\" (UID: \"086c327e-0e66-4f9c-83fb-85f688ffa253\") " pod="openshift-ingress/payload-processing-e2e-api-a-90db52-57478fddb7-vq79w"
Oct 31 21:33:21.577165 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:21.577124    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wn2kh\" (UniqueName: \"kubernetes.io/projected/086c327e-0e66-4f9c-83fb-85f688ffa253-kube-api-access-wn2kh\") pod \"payload-processing-e2e-api-a-90db52-57478fddb7-vq79w\" (UID: \"086c327e-0e66-4f9c-83fb-85f688ffa253\") " pod="openshift-ingress/payload-processing-e2e-api-a-90db52-57478fddb7-vq79w"
Oct 31 21:33:21.726714 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:21.726625    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-90db52-57f7c9d8dc-ltfg6"
Oct 31 21:33:21.805263 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:21.805231    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-90db52-57478fddb7-vq79w"
Oct 31 21:33:22.036013 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:22.035987    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-90db52-57f7c9d8dc-ltfg6"]
Oct 31 21:33:22.036626 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:33:22.036597    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9afe6453_662b_4786_8f53_b9d6d720750a.slice/crio-60a4517cb5a5a8a83be354c48847cefdb3b0129a35a2fe5d5675e9d249055375 WatchSource:0}: Error finding container 60a4517cb5a5a8a83be354c48847cefdb3b0129a35a2fe5d5675e9d249055375: Status 404 returned error can't find the container with id 60a4517cb5a5a8a83be354c48847cefdb3b0129a35a2fe5d5675e9d249055375
Oct 31 21:33:22.129643 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:22.129609    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-90db52-57f7c9d8dc-ltfg6" event={"ID":"9afe6453-662b-4786-8f53-b9d6d720750a","Type":"ContainerStarted","Data":"5be6111b0c923c5236b266b2bbdd8f14068af20fe2c84aed8f5de0c87d1af5cf"}
Oct 31 21:33:22.129807 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:22.129666    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-90db52-57f7c9d8dc-ltfg6" event={"ID":"9afe6453-662b-4786-8f53-b9d6d720750a","Type":"ContainerStarted","Data":"60a4517cb5a5a8a83be354c48847cefdb3b0129a35a2fe5d5675e9d249055375"}
Oct 31 21:33:22.129807 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:22.129688    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-90db52-57478fddb7-vq79w"]
Oct 31 21:33:22.129882 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:22.129828    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-90db52-57f7c9d8dc-ltfg6"
Oct 31 21:33:22.138248 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:33:22.138210    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod086c327e_0e66_4f9c_83fb_85f688ffa253.slice/crio-2e9968b15b6440ce6a6b0876f518b448017c5ddb461be3b6f815204b3d5ef81e WatchSource:0}: Error finding container 2e9968b15b6440ce6a6b0876f518b448017c5ddb461be3b6f815204b3d5ef81e: Status 404 returned error can't find the container with id 2e9968b15b6440ce6a6b0876f518b448017c5ddb461be3b6f815204b3d5ef81e
Oct 31 21:33:22.175741 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:22.175692    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-90db52-57f7c9d8dc-ltfg6" podStartSLOduration=1.175677458 podStartE2EDuration="1.175677458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:33:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:33:22.172765736 +0000 UTC m=+1042.089075166" watchObservedRunningTime="2026-10-31 21:33:22.175677458 +0000 UTC m=+1042.091986948"
Oct 31 21:33:22.604004 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:22.603928    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:22.902115 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:22.902024    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:23.030367 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:23.030328    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:23.069003 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:23.068971    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-084d807f-5ff89dd6bc-6hnlc"]
Oct 31 21:33:23.073163 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:23.073146    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-084d807f-5ff89dd6bc-6hnlc"
Oct 31 21:33:23.093549 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:23.093507    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-084d807f-5ff89dd6bc-6hnlc"]
Oct 31 21:33:23.105960 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:23.103679    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-084d807f-dockercfg-xhswz\""
Oct 31 21:33:23.105960 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:23.103848    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-084d807f\""
Oct 31 21:33:23.176017 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:23.173067    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-90db52-57478fddb7-vq79w" event={"ID":"086c327e-0e66-4f9c-83fb-85f688ffa253","Type":"ContainerStarted","Data":"8a4e597c08993bf07073ad5a1ad2a89ed76c6e064b1488737c30d545f4156a78"}
Oct 31 21:33:23.176017 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:23.173149    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-90db52-57478fddb7-vq79w" event={"ID":"086c327e-0e66-4f9c-83fb-85f688ffa253","Type":"ContainerStarted","Data":"2e9968b15b6440ce6a6b0876f518b448017c5ddb461be3b6f815204b3d5ef81e"}
Oct 31 21:33:23.176017 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:23.173221    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-90db52-57478fddb7-vq79w"
Oct 31 21:33:23.222963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:23.219033    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7gb2\" (UniqueName: \"kubernetes.io/projected/3c2ada97-edd3-47ec-9d4a-d0c48674685a-kube-api-access-f7gb2\") pod \"payload-pre-processing-e2e-mt-084d807f-5ff89dd6bc-6hnlc\" (UID: \"3c2ada97-edd3-47ec-9d4a-d0c48674685a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-084d807f-5ff89dd6bc-6hnlc"
Oct 31 21:33:23.222963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:23.219331    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c2ada97-edd3-47ec-9d4a-d0c48674685a-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-084d807f-5ff89dd6bc-6hnlc\" (UID: \"3c2ada97-edd3-47ec-9d4a-d0c48674685a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-084d807f-5ff89dd6bc-6hnlc"
Oct 31 21:33:23.271964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:23.268306    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-084d807f-7648554b69-6k7t8"]
Oct 31 21:33:23.285977 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:23.276356    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-084d807f-7648554b69-6k7t8"
Oct 31 21:33:23.315995 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:23.306293    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-90db52-57478fddb7-vq79w" podStartSLOduration=2.306271888 podStartE2EDuration="2.306271888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:33:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:33:23.2253501 +0000 UTC m=+1043.141659535" watchObservedRunningTime="2026-10-31 21:33:23.306271888 +0000 UTC m=+1043.222581314"
Oct 31 21:33:23.315995 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:23.310637    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-084d807f-7648554b69-6k7t8"]
Oct 31 21:33:23.328962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:23.325390    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7gb2\" (UniqueName: \"kubernetes.io/projected/3c2ada97-edd3-47ec-9d4a-d0c48674685a-kube-api-access-f7gb2\") pod \"payload-pre-processing-e2e-mt-084d807f-5ff89dd6bc-6hnlc\" (UID: \"3c2ada97-edd3-47ec-9d4a-d0c48674685a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-084d807f-5ff89dd6bc-6hnlc"
Oct 31 21:33:23.328962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:23.325663    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c2ada97-edd3-47ec-9d4a-d0c48674685a-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-084d807f-5ff89dd6bc-6hnlc\" (UID: \"3c2ada97-edd3-47ec-9d4a-d0c48674685a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-084d807f-5ff89dd6bc-6hnlc"
Oct 31 21:33:23.339764 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:23.339723    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c2ada97-edd3-47ec-9d4a-d0c48674685a-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-084d807f-5ff89dd6bc-6hnlc\" (UID: \"3c2ada97-edd3-47ec-9d4a-d0c48674685a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-084d807f-5ff89dd6bc-6hnlc"
Oct 31 21:33:23.430005 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:23.427036    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7gb2\" (UniqueName: \"kubernetes.io/projected/3c2ada97-edd3-47ec-9d4a-d0c48674685a-kube-api-access-f7gb2\") pod \"payload-pre-processing-e2e-mt-084d807f-5ff89dd6bc-6hnlc\" (UID: \"3c2ada97-edd3-47ec-9d4a-d0c48674685a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-084d807f-5ff89dd6bc-6hnlc"
Oct 31 21:33:23.435963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:23.431916    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gztpf\" (UniqueName: \"kubernetes.io/projected/069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3-kube-api-access-gztpf\") pod \"payload-processing-e2e-mt-084d807f-7648554b69-6k7t8\" (UID: \"069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3\") " pod="openshift-ingress/payload-processing-e2e-mt-084d807f-7648554b69-6k7t8"
Oct 31 21:33:23.435963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:23.432124    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3-plugins-config-volume\") pod \"payload-processing-e2e-mt-084d807f-7648554b69-6k7t8\" (UID: \"069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3\") " pod="openshift-ingress/payload-processing-e2e-mt-084d807f-7648554b69-6k7t8"
Oct 31 21:33:23.534975 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:23.534876    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gztpf\" (UniqueName: \"kubernetes.io/projected/069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3-kube-api-access-gztpf\") pod \"payload-processing-e2e-mt-084d807f-7648554b69-6k7t8\" (UID: \"069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3\") " pod="openshift-ingress/payload-processing-e2e-mt-084d807f-7648554b69-6k7t8"
Oct 31 21:33:23.535506 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:23.535476    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3-plugins-config-volume\") pod \"payload-processing-e2e-mt-084d807f-7648554b69-6k7t8\" (UID: \"069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3\") " pod="openshift-ingress/payload-processing-e2e-mt-084d807f-7648554b69-6k7t8"
Oct 31 21:33:23.537088 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:23.537066    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3-plugins-config-volume\") pod \"payload-processing-e2e-mt-084d807f-7648554b69-6k7t8\" (UID: \"069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3\") " pod="openshift-ingress/payload-processing-e2e-mt-084d807f-7648554b69-6k7t8"
Oct 31 21:33:23.560740 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:23.560713    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gztpf\" (UniqueName: \"kubernetes.io/projected/069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3-kube-api-access-gztpf\") pod \"payload-processing-e2e-mt-084d807f-7648554b69-6k7t8\" (UID: \"069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3\") " pod="openshift-ingress/payload-processing-e2e-mt-084d807f-7648554b69-6k7t8"
Oct 31 21:33:23.676605 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:23.676571    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-084d807f-7648554b69-6k7t8"
Oct 31 21:33:23.692417 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:23.692348    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-084d807f-5ff89dd6bc-6hnlc"
Oct 31 21:33:23.985681 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:23.985647    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-084d807f-7648554b69-6k7t8"]
Oct 31 21:33:23.986066 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:33:23.986042    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod069adf89_8ed5_4b2b_b5d1_ea0df0eea3e3.slice/crio-c45555d8d893a606e05fbd7d4fcc71c299c28d7570c270883f565b4dae97f7c5 WatchSource:0}: Error finding container c45555d8d893a606e05fbd7d4fcc71c299c28d7570c270883f565b4dae97f7c5: Status 404 returned error can't find the container with id c45555d8d893a606e05fbd7d4fcc71c299c28d7570c270883f565b4dae97f7c5
Oct 31 21:33:24.013452 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:24.013425    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-084d807f-5ff89dd6bc-6hnlc"]
Oct 31 21:33:24.016206 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:33:24.016180    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c2ada97_edd3_47ec_9d4a_d0c48674685a.slice/crio-f8ca8c8524e09c8a9c6c51b3ca06a1a979796aa2df0131fa2af239c96c1bc55e WatchSource:0}: Error finding container f8ca8c8524e09c8a9c6c51b3ca06a1a979796aa2df0131fa2af239c96c1bc55e: Status 404 returned error can't find the container with id f8ca8c8524e09c8a9c6c51b3ca06a1a979796aa2df0131fa2af239c96c1bc55e
Oct 31 21:33:24.161427 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:24.161388    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-084d807f-5ff89dd6bc-6hnlc" event={"ID":"3c2ada97-edd3-47ec-9d4a-d0c48674685a","Type":"ContainerStarted","Data":"f8b8866a9578a95563b791f01222eac672f7eb6a9cb1d585a95915d91ad32496"}
Oct 31 21:33:24.161627 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:24.161456    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-084d807f-5ff89dd6bc-6hnlc"
Oct 31 21:33:24.161627 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:24.161483    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-084d807f-5ff89dd6bc-6hnlc" event={"ID":"3c2ada97-edd3-47ec-9d4a-d0c48674685a","Type":"ContainerStarted","Data":"f8ca8c8524e09c8a9c6c51b3ca06a1a979796aa2df0131fa2af239c96c1bc55e"}
Oct 31 21:33:24.163008 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:24.162977    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-084d807f-7648554b69-6k7t8" event={"ID":"069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3","Type":"ContainerStarted","Data":"39f7762dc53cbf5e562fff9e6a49aecaef1c6612a041064278613d962edebc67"}
Oct 31 21:33:24.163133 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:24.163027    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-084d807f-7648554b69-6k7t8" event={"ID":"069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3","Type":"ContainerStarted","Data":"c45555d8d893a606e05fbd7d4fcc71c299c28d7570c270883f565b4dae97f7c5"}
Oct 31 21:33:24.163188 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:24.163170    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-084d807f-7648554b69-6k7t8"
Oct 31 21:33:24.205749 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:24.205697    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-084d807f-5ff89dd6bc-6hnlc" podStartSLOduration=1.205683613 podStartE2EDuration="1.205683613s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:33:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:33:24.205481435 +0000 UTC m=+1044.121790879" watchObservedRunningTime="2026-10-31 21:33:24.205683613 +0000 UTC m=+1044.121993044"
Oct 31 21:33:24.249975 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:24.249826    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-084d807f-7648554b69-6k7t8" podStartSLOduration=1.249812425 podStartE2EDuration="1.249812425s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:33:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:33:24.248416501 +0000 UTC m=+1044.164725932" watchObservedRunningTime="2026-10-31 21:33:24.249812425 +0000 UTC m=+1044.166121855"
Oct 31 21:33:24.324365 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:24.324331    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:26.027010 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:26.026976    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:27.755926 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:27.755887    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:29.387176 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:29.387134    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:29.634982 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:29.632064    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-697b9f5476-zz4qf"]
Oct 31 21:33:29.646017 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:29.639141    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-697b9f5476-zz4qf"
Oct 31 21:33:29.663921 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:29.661973    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-90db52-serving-cert\""
Oct 31 21:33:29.666677 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:29.666638    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-697b9f5476-zz4qf"]
Oct 31 21:33:29.776996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:29.773041    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-maas-api-tls\") pod \"maas-api-e2e-api-a-90db52-697b9f5476-zz4qf\" (UID: \"9f6b4afa-aaec-48fd-be70-7e69f1d74fc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-697b9f5476-zz4qf"
Oct 31 21:33:29.776996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:29.773113    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqmgt\" (UniqueName: \"kubernetes.io/projected/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-kube-api-access-sqmgt\") pod \"maas-api-e2e-api-a-90db52-697b9f5476-zz4qf\" (UID: \"9f6b4afa-aaec-48fd-be70-7e69f1d74fc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-697b9f5476-zz4qf"
Oct 31 21:33:29.776996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:29.773203    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-openshift-service-ca\") pod \"maas-api-e2e-api-a-90db52-697b9f5476-zz4qf\" (UID: \"9f6b4afa-aaec-48fd-be70-7e69f1d74fc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-697b9f5476-zz4qf"
Oct 31 21:33:29.776996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:29.773320    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-metrics-certs\") pod \"maas-api-e2e-api-a-90db52-697b9f5476-zz4qf\" (UID: \"9f6b4afa-aaec-48fd-be70-7e69f1d74fc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-697b9f5476-zz4qf"
Oct 31 21:33:29.813490 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:29.810351    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:29.897010 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:29.896020    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-maas-api-tls\") pod \"maas-api-e2e-api-a-90db52-697b9f5476-zz4qf\" (UID: \"9f6b4afa-aaec-48fd-be70-7e69f1d74fc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-697b9f5476-zz4qf"
Oct 31 21:33:29.897010 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:29.896088    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqmgt\" (UniqueName: \"kubernetes.io/projected/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-kube-api-access-sqmgt\") pod \"maas-api-e2e-api-a-90db52-697b9f5476-zz4qf\" (UID: \"9f6b4afa-aaec-48fd-be70-7e69f1d74fc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-697b9f5476-zz4qf"
Oct 31 21:33:29.897010 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:29.896189    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-openshift-service-ca\") pod \"maas-api-e2e-api-a-90db52-697b9f5476-zz4qf\" (UID: \"9f6b4afa-aaec-48fd-be70-7e69f1d74fc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-697b9f5476-zz4qf"
Oct 31 21:33:29.897010 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:29.896316    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-metrics-certs\") pod \"maas-api-e2e-api-a-90db52-697b9f5476-zz4qf\" (UID: \"9f6b4afa-aaec-48fd-be70-7e69f1d74fc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-697b9f5476-zz4qf"
Oct 31 21:33:29.912972 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:33:29.902111    4460 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-serving-cert: secret "maas-api-e2e-api-a-90db52-serving-cert" not found
Oct 31 21:33:29.912972 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:33:29.902222    4460 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-maas-api-tls podName:9f6b4afa-aaec-48fd-be70-7e69f1d74fc1 nodeName:}" failed. No retries permitted until 2026-10-31 21:33:30.402192042 +0000 UTC m=+1050.318501475 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-maas-api-tls") pod "maas-api-e2e-api-a-90db52-697b9f5476-zz4qf" (UID: "9f6b4afa-aaec-48fd-be70-7e69f1d74fc1") : secret "maas-api-e2e-api-a-90db52-serving-cert" not found
Oct 31 21:33:29.912972 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:29.904223    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-openshift-service-ca\") pod \"maas-api-e2e-api-a-90db52-697b9f5476-zz4qf\" (UID: \"9f6b4afa-aaec-48fd-be70-7e69f1d74fc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-697b9f5476-zz4qf"
Oct 31 21:33:30.065272 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:30.065060    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:30.179031 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:30.139528    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-metrics-certs\") pod \"maas-api-e2e-api-a-90db52-697b9f5476-zz4qf\" (UID: \"9f6b4afa-aaec-48fd-be70-7e69f1d74fc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-697b9f5476-zz4qf"
Oct 31 21:33:30.182970 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:30.179320    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqmgt\" (UniqueName: \"kubernetes.io/projected/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-kube-api-access-sqmgt\") pod \"maas-api-e2e-api-a-90db52-697b9f5476-zz4qf\" (UID: \"9f6b4afa-aaec-48fd-be70-7e69f1d74fc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-697b9f5476-zz4qf"
Oct 31 21:33:30.466017 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:30.465905    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-maas-api-tls\") pod \"maas-api-e2e-api-a-90db52-697b9f5476-zz4qf\" (UID: \"9f6b4afa-aaec-48fd-be70-7e69f1d74fc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-697b9f5476-zz4qf"
Oct 31 21:33:30.466499 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:33:30.466053    4460 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-serving-cert: secret "maas-api-e2e-api-a-90db52-serving-cert" not found
Oct 31 21:33:30.466499 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:33:30.466217    4460 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-maas-api-tls podName:9f6b4afa-aaec-48fd-be70-7e69f1d74fc1 nodeName:}" failed. No retries permitted until 2026-10-31 21:33:31.466194855 +0000 UTC m=+1051.382504277 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-maas-api-tls") pod "maas-api-e2e-api-a-90db52-697b9f5476-zz4qf" (UID: "9f6b4afa-aaec-48fd-be70-7e69f1d74fc1") : secret "maas-api-e2e-api-a-90db52-serving-cert" not found
Oct 31 21:33:30.526749 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:30.526657    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:30.644306 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:30.644134    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:31.512394 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:31.512355    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-maas-api-tls\") pod \"maas-api-e2e-api-a-90db52-697b9f5476-zz4qf\" (UID: \"9f6b4afa-aaec-48fd-be70-7e69f1d74fc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-697b9f5476-zz4qf"
Oct 31 21:33:31.514778 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:31.514749    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-maas-api-tls\") pod \"maas-api-e2e-api-a-90db52-697b9f5476-zz4qf\" (UID: \"9f6b4afa-aaec-48fd-be70-7e69f1d74fc1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-697b9f5476-zz4qf"
Oct 31 21:33:31.633284 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:31.633246    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-697b9f5476-zz4qf"
Oct 31 21:33:31.920807 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:31.920721    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:31.944126 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:31.944102    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-697b9f5476-zz4qf"]
Oct 31 21:33:31.944591 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:33:31.944572    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f6b4afa_aaec_48fd_be70_7e69f1d74fc1.slice/crio-019726a245d618d0ae978596efeb02a524a17aa81241843a5d65a13f172e5bca WatchSource:0}: Error finding container 019726a245d618d0ae978596efeb02a524a17aa81241843a5d65a13f172e5bca: Status 404 returned error can't find the container with id 019726a245d618d0ae978596efeb02a524a17aa81241843a5d65a13f172e5bca
Oct 31 21:33:31.945996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:31.945979    4460 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 21:33:32.281272 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:32.281239    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-697b9f5476-zz4qf" event={"ID":"9f6b4afa-aaec-48fd-be70-7e69f1d74fc1","Type":"ContainerStarted","Data":"019726a245d618d0ae978596efeb02a524a17aa81241843a5d65a13f172e5bca"}
Oct 31 21:33:33.156888 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:33.156861    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-90db52-57f7c9d8dc-ltfg6"
Oct 31 21:33:33.289750 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:33.289711    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-697b9f5476-zz4qf" event={"ID":"9f6b4afa-aaec-48fd-be70-7e69f1d74fc1","Type":"ContainerStarted","Data":"a8119e65258091e928f07ef43bffd347dd8690b4e22b343f63b978a6a077fe69"}
Oct 31 21:33:33.289919 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:33.289780    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-697b9f5476-zz4qf"
Oct 31 21:33:33.342175 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:33.342117    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-697b9f5476-zz4qf" podStartSLOduration=3.8356044110000003 podStartE2EDuration="4.342103163s" totalImagesPullingTime="506.498752ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:33:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:33:33.338930601 +0000 UTC m=+1053.255240042" watchObservedRunningTime="2026-10-31 21:33:33.342103163 +0000 UTC m=+1053.258412593"
Oct 31 21:33:33.688689 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:33.688648    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:33.909669 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:33.909632    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:34.022503 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:34.022470    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:34.444548 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:34.444470    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:34.571291 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:34.571254    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:35.172090 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:35.172057    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-90db52-57478fddb7-vq79w"
Oct 31 21:33:35.172264 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:35.172151    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-084d807f-5ff89dd6bc-6hnlc"
Oct 31 21:33:35.172366 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:35.172348    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-084d807f-7648554b69-6k7t8"
Oct 31 21:33:35.723873 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:35.723822    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:36.734639 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:36.734597    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:36.847921 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:36.847886    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:37.100968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:37.100015    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7ac3c8-76bc49454f-vfdpw"]
Oct 31 21:33:37.195264 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:37.195229    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7ac3c8-76bc49454f-vfdpw"]
Oct 31 21:33:37.195579 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:37.195560    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-7ac3c8-76bc49454f-vfdpw"
Oct 31 21:33:37.230218 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:37.230183    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-7ac3c8\""
Oct 31 21:33:37.230579 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:37.230274    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-7ac3c8-dockercfg-mxj8v\""
Oct 31 21:33:37.374330 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:37.374229    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8497c5c1-966e-48c3-beef-d982c8ab7627-plugins-config-volume\") pod \"payload-processing-e2e-api-a-7ac3c8-76bc49454f-vfdpw\" (UID: \"8497c5c1-966e-48c3-beef-d982c8ab7627\") " pod="openshift-ingress/payload-processing-e2e-api-a-7ac3c8-76bc49454f-vfdpw"
Oct 31 21:33:37.374561 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:37.374539    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxrw8\" (UniqueName: \"kubernetes.io/projected/8497c5c1-966e-48c3-beef-d982c8ab7627-kube-api-access-rxrw8\") pod \"payload-processing-e2e-api-a-7ac3c8-76bc49454f-vfdpw\" (UID: \"8497c5c1-966e-48c3-beef-d982c8ab7627\") " pod="openshift-ingress/payload-processing-e2e-api-a-7ac3c8-76bc49454f-vfdpw"
Oct 31 21:33:37.475974 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:37.475916    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8497c5c1-966e-48c3-beef-d982c8ab7627-plugins-config-volume\") pod \"payload-processing-e2e-api-a-7ac3c8-76bc49454f-vfdpw\" (UID: \"8497c5c1-966e-48c3-beef-d982c8ab7627\") " pod="openshift-ingress/payload-processing-e2e-api-a-7ac3c8-76bc49454f-vfdpw"
Oct 31 21:33:37.476158 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:37.476074    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxrw8\" (UniqueName: \"kubernetes.io/projected/8497c5c1-966e-48c3-beef-d982c8ab7627-kube-api-access-rxrw8\") pod \"payload-processing-e2e-api-a-7ac3c8-76bc49454f-vfdpw\" (UID: \"8497c5c1-966e-48c3-beef-d982c8ab7627\") " pod="openshift-ingress/payload-processing-e2e-api-a-7ac3c8-76bc49454f-vfdpw"
Oct 31 21:33:37.476663 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:37.476642    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8497c5c1-966e-48c3-beef-d982c8ab7627-plugins-config-volume\") pod \"payload-processing-e2e-api-a-7ac3c8-76bc49454f-vfdpw\" (UID: \"8497c5c1-966e-48c3-beef-d982c8ab7627\") " pod="openshift-ingress/payload-processing-e2e-api-a-7ac3c8-76bc49454f-vfdpw"
Oct 31 21:33:37.503676 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:37.503639    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxrw8\" (UniqueName: \"kubernetes.io/projected/8497c5c1-966e-48c3-beef-d982c8ab7627-kube-api-access-rxrw8\") pod \"payload-processing-e2e-api-a-7ac3c8-76bc49454f-vfdpw\" (UID: \"8497c5c1-966e-48c3-beef-d982c8ab7627\") " pod="openshift-ingress/payload-processing-e2e-api-a-7ac3c8-76bc49454f-vfdpw"
Oct 31 21:33:37.524579 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:37.524526    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:37.528424 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:37.528401    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-7ac3c8-76bc49454f-vfdpw"
Oct 31 21:33:37.908733 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:37.908701    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7ac3c8-76bc49454f-vfdpw"]
Oct 31 21:33:37.909243 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:33:37.909218    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8497c5c1_966e_48c3_beef_d982c8ab7627.slice/crio-95ad3b429208d154c5ecc1f5e4854dfc871c061828b8493825acf8ec91c546fa WatchSource:0}: Error finding container 95ad3b429208d154c5ecc1f5e4854dfc871c061828b8493825acf8ec91c546fa: Status 404 returned error can't find the container with id 95ad3b429208d154c5ecc1f5e4854dfc871c061828b8493825acf8ec91c546fa
Oct 31 21:33:38.399651 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:38.399609    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-7ac3c8-76bc49454f-vfdpw" event={"ID":"8497c5c1-966e-48c3-beef-d982c8ab7627","Type":"ContainerStarted","Data":"ad8fea4da5a956400f0441e12c1dd1bcd7d1c33fb7d78620b3ca63bb0183024f"}
Oct 31 21:33:38.399955 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:38.399917    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-7ac3c8-76bc49454f-vfdpw" event={"ID":"8497c5c1-966e-48c3-beef-d982c8ab7627","Type":"ContainerStarted","Data":"95ad3b429208d154c5ecc1f5e4854dfc871c061828b8493825acf8ec91c546fa"}
Oct 31 21:33:38.401803 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:38.401766    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-7ac3c8-76bc49454f-vfdpw"
Oct 31 21:33:38.447995 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:38.447929    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-7ac3c8-76bc49454f-vfdpw" podStartSLOduration=2.447915148 podStartE2EDuration="2.447915148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:33:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:33:38.446847482 +0000 UTC m=+1058.363156914" watchObservedRunningTime="2026-10-31 21:33:38.447915148 +0000 UTC m=+1058.364224577"
Oct 31 21:33:39.246999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:39.246969    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:39.328833 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:39.328802    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-697b9f5476-zz4qf"
Oct 31 21:33:40.919918 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:40.919879    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:42.661287 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:42.661256    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:42.776764 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:42.776712    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:33:44.777888 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:44.777851    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:45.604420 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:45.604381    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:47.437558 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:47.437517    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:49.411933 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.411901    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-7ac3c8-76bc49454f-vfdpw"
Oct 31 21:33:49.445231 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.445182    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:49.661812 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.658531    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"]
Oct 31 21:33:49.677995 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.677877    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:33:49.697406 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.697378    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"]
Oct 31 21:33:49.704624 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.704580    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"]
Oct 31 21:33:49.710356 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.710302    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:33:49.721107 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.721072    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"]
Oct 31 21:33:49.820284 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.820244    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istio-podinfo\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:33:49.820488 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.820307    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-credential-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:33:49.820488 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.820390    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-workload-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:33:49.820583 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.820558    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istio-envoy\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:33:49.820622 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.820608    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-workload-certs\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:33:49.820673 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.820653    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-workload-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:33:49.820746 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.820707    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istio-token\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:33:49.820839 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.820816    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-istio-envoy\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:33:49.820890 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.820854    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-credential-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:33:49.820979 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.820934    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9d9464c-85ec-4323-81b6-7201345e9f07-istiod-ca-cert\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:33:49.821030 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.821010    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-workload-certs\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:33:49.821174 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.821151    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9d9464c-85ec-4323-81b6-7201345e9f07-istio-token\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:33:49.821320 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.821203    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq9xh\" (UniqueName: \"kubernetes.io/projected/f9d9464c-85ec-4323-81b6-7201345e9f07-kube-api-access-mq9xh\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:33:49.821320 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.821241    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istiod-ca-cert\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:33:49.821401 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.821369    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istio-data\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:33:49.821459 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.821446    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-istio-data\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:33:49.821563 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.821546    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ztxn\" (UniqueName: \"kubernetes.io/projected/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-kube-api-access-7ztxn\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:33:49.821626 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.821581    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9d9464c-85ec-4323-81b6-7201345e9f07-istio-podinfo\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:33:49.923973 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.923918    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istio-podinfo\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:33:49.924164 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.924003    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-credential-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:33:49.924164 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.924026    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-workload-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:33:49.924164 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.924049    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istio-envoy\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:33:49.924164 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.924069    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-workload-certs\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:33:49.924164 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.924104    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-workload-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:33:49.924164 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.924135    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istio-token\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:33:49.924560 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.924166    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-istio-envoy\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:33:49.924560 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.924219    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-credential-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:33:49.924560 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.924258    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9d9464c-85ec-4323-81b6-7201345e9f07-istiod-ca-cert\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:33:49.924560 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.924301    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-workload-certs\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:33:49.924560 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.924345    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9d9464c-85ec-4323-81b6-7201345e9f07-istio-token\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:33:49.924560 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.924399    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mq9xh\" (UniqueName: \"kubernetes.io/projected/f9d9464c-85ec-4323-81b6-7201345e9f07-kube-api-access-mq9xh\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:33:49.924560 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.924453    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-workload-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:33:49.924560 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.924459    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istiod-ca-cert\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:33:49.924560 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.924511    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-credential-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:33:49.924560 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.924535    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-workload-certs\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:33:49.924560 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.924554    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istio-data\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:33:49.925172 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.924639    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-istio-data\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:33:49.925172 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.924685    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ztxn\" (UniqueName: \"kubernetes.io/projected/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-kube-api-access-7ztxn\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:33:49.925172 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.924718    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9d9464c-85ec-4323-81b6-7201345e9f07-istio-podinfo\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:33:49.925172 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.925133    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9d9464c-85ec-4323-81b6-7201345e9f07-istiod-ca-cert\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:33:49.926231 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.925552    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-workload-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:33:49.926231 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.925748    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-workload-certs\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:33:49.926231 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.925802    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-credential-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:33:49.926231 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.925848    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-istio-data\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:33:49.926231 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.926191    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istio-data\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:33:49.926719 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.926685    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istiod-ca-cert\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:33:49.926891 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.926865    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-istio-envoy\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:33:49.927207 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.927186    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istio-podinfo\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:33:49.927403 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.927385    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istio-envoy\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:33:49.927664 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.927646    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9d9464c-85ec-4323-81b6-7201345e9f07-istio-podinfo\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:33:49.951996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.951872    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istio-token\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:33:49.952140 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.952005    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ztxn\" (UniqueName: \"kubernetes.io/projected/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-kube-api-access-7ztxn\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:33:49.952140 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.952009    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9d9464c-85ec-4323-81b6-7201345e9f07-istio-token\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:33:49.952218 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:49.952151    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq9xh\" (UniqueName: \"kubernetes.io/projected/f9d9464c-85ec-4323-81b6-7201345e9f07-kube-api-access-mq9xh\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:33:50.031636 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.031590    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:33:50.040258 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.040227    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:33:50.059920 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.059820    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:50.171518 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.171477    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:50.356026 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.350863    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:50.398477 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.398443    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"]
Oct 31 21:33:50.399319 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:33:50.399284    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffaf1375_c974_4e1d_a8bd_b880d7ea6e96.slice/crio-e9d2ff52d274093581f4e0694871afc5628dece656a812e7936c26f02ed55907 WatchSource:0}: Error finding container e9d2ff52d274093581f4e0694871afc5628dece656a812e7936c26f02ed55907: Status 404 returned error can't find the container with id e9d2ff52d274093581f4e0694871afc5628dece656a812e7936c26f02ed55907
Oct 31 21:33:50.402058 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.402024    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:33:50.402136 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.402101    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:33:50.402136 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.402128    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:33:50.422916 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.422889    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"]
Oct 31 21:33:50.424561 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:33:50.424534    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9d9464c_85ec_4323_81b6_7201345e9f07.slice/crio-74acce2f92d2b5d8ef9fabf530c9442d95af220008c63c92d258c8521afb816c WatchSource:0}: Error finding container 74acce2f92d2b5d8ef9fabf530c9442d95af220008c63c92d258c8521afb816c: Status 404 returned error can't find the container with id 74acce2f92d2b5d8ef9fabf530c9442d95af220008c63c92d258c8521afb816c
Oct 31 21:33:50.427442 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.427410    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:33:50.427521 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.427485    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:33:50.427521 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.427515    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:33:50.492499 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.492401    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh" event={"ID":"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96","Type":"ContainerStarted","Data":"e9d2ff52d274093581f4e0694871afc5628dece656a812e7936c26f02ed55907"}
Oct 31 21:33:50.494687 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.494657    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm" event={"ID":"f9d9464c-85ec-4323-81b6-7201345e9f07","Type":"ContainerStarted","Data":"74acce2f92d2b5d8ef9fabf530c9442d95af220008c63c92d258c8521afb816c"}
Oct 31 21:33:50.595588 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.595545    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"]
Oct 31 21:33:50.606018 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.604059    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:33:50.626468 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.624869    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-371531-openshift-default-dockercfg-kvqjw\""
Oct 31 21:33:50.626468 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.624897    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"]
Oct 31 21:33:50.653439 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.653390    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:50.749693 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.749652    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-workload-certs\") pod \"e2e-api-b-371531-openshift-default-d977d868c-nzsxr\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") " pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:33:50.752620 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.752591    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-istio-data\") pod \"e2e-api-b-371531-openshift-default-d977d868c-nzsxr\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") " pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:33:50.752894 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.752874    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-istio-envoy\") pod \"e2e-api-b-371531-openshift-default-d977d868c-nzsxr\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") " pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:33:50.753122 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.753103    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-credential-socket\") pod \"e2e-api-b-371531-openshift-default-d977d868c-nzsxr\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") " pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:33:50.753325 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.753308    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-workload-socket\") pod \"e2e-api-b-371531-openshift-default-d977d868c-nzsxr\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") " pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:33:50.753507 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.753490    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8842af32-490d-40d9-9da3-e9daba253252-istio-podinfo\") pod \"e2e-api-b-371531-openshift-default-d977d868c-nzsxr\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") " pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:33:50.753688 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.753671    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8842af32-490d-40d9-9da3-e9daba253252-istiod-ca-cert\") pod \"e2e-api-b-371531-openshift-default-d977d868c-nzsxr\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") " pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:33:50.753903 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.753886    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8842af32-490d-40d9-9da3-e9daba253252-istio-token\") pod \"e2e-api-b-371531-openshift-default-d977d868c-nzsxr\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") " pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:33:50.754091 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.754072    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9b5dg\" (UniqueName: \"kubernetes.io/projected/8842af32-490d-40d9-9da3-e9daba253252-kube-api-access-9b5dg\") pod \"e2e-api-b-371531-openshift-default-d977d868c-nzsxr\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") " pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:33:50.872022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.846561    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:50.948964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.922249    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-workload-certs\") pod \"e2e-api-b-371531-openshift-default-d977d868c-nzsxr\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") " pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:33:50.948964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.922421    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-istio-data\") pod \"e2e-api-b-371531-openshift-default-d977d868c-nzsxr\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") " pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:33:50.948964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.922462    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-istio-envoy\") pod \"e2e-api-b-371531-openshift-default-d977d868c-nzsxr\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") " pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:33:50.948964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.922534    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-credential-socket\") pod \"e2e-api-b-371531-openshift-default-d977d868c-nzsxr\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") " pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:33:50.948964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.922603    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-workload-socket\") pod \"e2e-api-b-371531-openshift-default-d977d868c-nzsxr\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") " pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:33:50.948964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.922659    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8842af32-490d-40d9-9da3-e9daba253252-istio-podinfo\") pod \"e2e-api-b-371531-openshift-default-d977d868c-nzsxr\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") " pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:33:50.948964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.922764    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8842af32-490d-40d9-9da3-e9daba253252-istiod-ca-cert\") pod \"e2e-api-b-371531-openshift-default-d977d868c-nzsxr\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") " pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:33:50.948964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.922878    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8842af32-490d-40d9-9da3-e9daba253252-istio-token\") pod \"e2e-api-b-371531-openshift-default-d977d868c-nzsxr\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") " pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:33:50.948964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.922909    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9b5dg\" (UniqueName: \"kubernetes.io/projected/8842af32-490d-40d9-9da3-e9daba253252-kube-api-access-9b5dg\") pod \"e2e-api-b-371531-openshift-default-d977d868c-nzsxr\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") " pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:33:50.948964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.923746    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-workload-certs\") pod \"e2e-api-b-371531-openshift-default-d977d868c-nzsxr\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") " pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:33:50.948964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.924035    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-istio-data\") pod \"e2e-api-b-371531-openshift-default-d977d868c-nzsxr\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") " pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:33:50.978788 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.968618    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-credential-socket\") pod \"e2e-api-b-371531-openshift-default-d977d868c-nzsxr\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") " pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:33:50.978788 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.969454    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8842af32-490d-40d9-9da3-e9daba253252-istiod-ca-cert\") pod \"e2e-api-b-371531-openshift-default-d977d868c-nzsxr\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") " pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:33:50.978788 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:50.969714    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-workload-socket\") pod \"e2e-api-b-371531-openshift-default-d977d868c-nzsxr\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") " pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:33:51.044963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:51.044560    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8842af32-490d-40d9-9da3-e9daba253252-istio-podinfo\") pod \"e2e-api-b-371531-openshift-default-d977d868c-nzsxr\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") " pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:33:51.052762 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:51.052676    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9b5dg\" (UniqueName: \"kubernetes.io/projected/8842af32-490d-40d9-9da3-e9daba253252-kube-api-access-9b5dg\") pod \"e2e-api-b-371531-openshift-default-d977d868c-nzsxr\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") " pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:33:51.054345 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:51.054315    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8842af32-490d-40d9-9da3-e9daba253252-istio-token\") pod \"e2e-api-b-371531-openshift-default-d977d868c-nzsxr\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") " pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:33:51.128336 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:51.096307    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-istio-envoy\") pod \"e2e-api-b-371531-openshift-default-d977d868c-nzsxr\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") " pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:33:51.226248 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:51.226214    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:33:51.455804 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:33:51.455696    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:33:51.636831 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:51.636792    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh" event={"ID":"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96","Type":"ContainerStarted","Data":"b78878097670787314777a4ee6046b93b29fb04701737b8d65fcdc0f7f43eb8f"}
Oct 31 21:33:51.657010 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:51.656965    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"]
Oct 31 21:33:51.657211 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:51.657061    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm" event={"ID":"f9d9464c-85ec-4323-81b6-7201345e9f07","Type":"ContainerStarted","Data":"bbb54219eb7004f0b8889566de9cd2a82fe6161b0d7931ba479dbc8e070227e7"}
Oct 31 21:33:51.673905 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:51.673867    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:33:51.674215 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:51.674186    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:33:51.674381 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:51.674363    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:33:51.693806 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:51.693752    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh" podStartSLOduration=2.693736736 podStartE2EDuration="2.693736736s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:33:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:33:51.689278217 +0000 UTC m=+1071.605587648" watchObservedRunningTime="2026-10-31 21:33:51.693736736 +0000 UTC m=+1071.610046167"
Oct 31 21:33:51.740651 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:51.740601    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm" podStartSLOduration=2.740584726 podStartE2EDuration="2.740584726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:33:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:33:51.740199378 +0000 UTC m=+1071.656508808" watchObservedRunningTime="2026-10-31 21:33:51.740584726 +0000 UTC m=+1071.656894156"
Oct 31 21:33:52.038965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:52.036154    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:33:52.047010 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:52.041342    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:33:52.050972 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:52.049126    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 31 21:33:52.050972 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:52.049202    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh" podUID="ffaf1375-c974-4e1d-a8bd-b880d7ea6e96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 31 21:33:52.050972 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:52.049553    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 31 21:33:52.050972 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:52.049590    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm" podUID="f9d9464c-85ec-4323-81b6-7201345e9f07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 31 21:33:52.441031 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:52.437110    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:52.693027 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:52.692209    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr" event={"ID":"8842af32-490d-40d9-9da3-e9daba253252","Type":"ContainerStarted","Data":"939d3519c52c12e7c628467a6e348dbbfd3ec477c78d265f11274b60d5cc3919"}
Oct 31 21:33:52.693027 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:52.692299    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr" event={"ID":"8842af32-490d-40d9-9da3-e9daba253252","Type":"ContainerStarted","Data":"42e97361970898880c3b09e7d5a7808e1548b3c584b98b5c4b3827e51204beb2"}
Oct 31 21:33:52.770420 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:52.768136    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr" podStartSLOduration=2.768116953 podStartE2EDuration="2.768116953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:33:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:33:52.766429832 +0000 UTC m=+1072.682739263" watchObservedRunningTime="2026-10-31 21:33:52.768116953 +0000 UTC m=+1072.684426386"
Oct 31 21:33:53.033961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:53.032375    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 31 21:33:53.033961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:53.032438    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh" podUID="ffaf1375-c974-4e1d-a8bd-b880d7ea6e96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 31 21:33:53.049967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:53.049402    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 31 21:33:53.049967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:53.049471    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm" podUID="f9d9464c-85ec-4323-81b6-7201345e9f07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 31 21:33:53.242963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:53.241547    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:33:53.257099 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:53.256725    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:33:53.689757 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:53.689724    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:33:53.690905 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:53.690885    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:33:54.034782 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:54.034730    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 31 21:33:54.035362 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:54.034827    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh" podUID="ffaf1375-c974-4e1d-a8bd-b880d7ea6e96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 31 21:33:54.042974 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:54.042490    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 31 21:33:54.042974 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:54.042565    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm" podUID="f9d9464c-85ec-4323-81b6-7201345e9f07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 31 21:33:54.717338 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:54.717303    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:55.036966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:55.034310    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 31 21:33:55.036966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:55.034391    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh" podUID="ffaf1375-c974-4e1d-a8bd-b880d7ea6e96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 31 21:33:55.059964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:55.059174    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 31 21:33:55.059964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:55.059248    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm" podUID="f9d9464c-85ec-4323-81b6-7201345e9f07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 31 21:33:55.172496 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:55.168658    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:55.282992 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:55.282959    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:55.657564 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:55.657467    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:56.046100 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:56.046058    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 31 21:33:56.046755 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:56.046718    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm" podUID="f9d9464c-85ec-4323-81b6-7201345e9f07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 31 21:33:56.047348 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:56.047322    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 31 21:33:56.047542 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:56.047518    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh" podUID="ffaf1375-c974-4e1d-a8bd-b880d7ea6e96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 31 21:33:57.008022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:57.007733    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:57.086148 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:57.086103    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 31 21:33:57.086574 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:57.086168    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm" podUID="f9d9464c-85ec-4323-81b6-7201345e9f07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 31 21:33:57.086574 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:57.086269    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 31 21:33:57.086574 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:57.086305    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh" podUID="ffaf1375-c974-4e1d-a8bd-b880d7ea6e96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 31 21:33:57.151967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:57.134611    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:57.404630 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:57.404590    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-371531-7b5cd97689-rtfrd"]
Oct 31 21:33:57.650970 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:57.637817    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-371531-7b5cd97689-rtfrd"]
Oct 31 21:33:57.650970 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:57.638030    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-371531-7b5cd97689-rtfrd"
Oct 31 21:33:57.712029 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:57.711485    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-371531-dockercfg-mpgp5\""
Oct 31 21:33:57.712029 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:57.711568    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-371531\""
Oct 31 21:33:57.784966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:57.770396    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vdwt\" (UniqueName: \"kubernetes.io/projected/9bcfc171-925d-4bfd-a2b6-61c8b90bc395-kube-api-access-6vdwt\") pod \"payload-processing-e2e-api-b-371531-7b5cd97689-rtfrd\" (UID: \"9bcfc171-925d-4bfd-a2b6-61c8b90bc395\") " pod="openshift-ingress/payload-processing-e2e-api-b-371531-7b5cd97689-rtfrd"
Oct 31 21:33:57.788966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:57.785500    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bcfc171-925d-4bfd-a2b6-61c8b90bc395-plugins-config-volume\") pod \"payload-processing-e2e-api-b-371531-7b5cd97689-rtfrd\" (UID: \"9bcfc171-925d-4bfd-a2b6-61c8b90bc395\") " pod="openshift-ingress/payload-processing-e2e-api-b-371531-7b5cd97689-rtfrd"
Oct 31 21:33:57.891172 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:57.891132    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bcfc171-925d-4bfd-a2b6-61c8b90bc395-plugins-config-volume\") pod \"payload-processing-e2e-api-b-371531-7b5cd97689-rtfrd\" (UID: \"9bcfc171-925d-4bfd-a2b6-61c8b90bc395\") " pod="openshift-ingress/payload-processing-e2e-api-b-371531-7b5cd97689-rtfrd"
Oct 31 21:33:57.891362 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:57.891315    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vdwt\" (UniqueName: \"kubernetes.io/projected/9bcfc171-925d-4bfd-a2b6-61c8b90bc395-kube-api-access-6vdwt\") pod \"payload-processing-e2e-api-b-371531-7b5cd97689-rtfrd\" (UID: \"9bcfc171-925d-4bfd-a2b6-61c8b90bc395\") " pod="openshift-ingress/payload-processing-e2e-api-b-371531-7b5cd97689-rtfrd"
Oct 31 21:33:57.892001 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:57.891975    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bcfc171-925d-4bfd-a2b6-61c8b90bc395-plugins-config-volume\") pod \"payload-processing-e2e-api-b-371531-7b5cd97689-rtfrd\" (UID: \"9bcfc171-925d-4bfd-a2b6-61c8b90bc395\") " pod="openshift-ingress/payload-processing-e2e-api-b-371531-7b5cd97689-rtfrd"
Oct 31 21:33:57.917039 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:57.916998    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vdwt\" (UniqueName: \"kubernetes.io/projected/9bcfc171-925d-4bfd-a2b6-61c8b90bc395-kube-api-access-6vdwt\") pod \"payload-processing-e2e-api-b-371531-7b5cd97689-rtfrd\" (UID: \"9bcfc171-925d-4bfd-a2b6-61c8b90bc395\") " pod="openshift-ingress/payload-processing-e2e-api-b-371531-7b5cd97689-rtfrd"
Oct 31 21:33:58.036818 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:58.036785    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 31 21:33:58.037150 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:58.037118    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh" podUID="ffaf1375-c974-4e1d-a8bd-b880d7ea6e96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 31 21:33:58.038880 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:58.038034    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:58.041599 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:58.041573    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 31 21:33:58.041722 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:58.041624    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm" podUID="f9d9464c-85ec-4323-81b6-7201345e9f07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 31 21:33:58.091000 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:58.090964    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-371531-7b5cd97689-rtfrd"
Oct 31 21:33:58.672119 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:58.672084    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-371531-7b5cd97689-rtfrd"]
Oct 31 21:33:58.674160 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:33:58.674127    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bcfc171_925d_4bfd_a2b6_61c8b90bc395.slice/crio-4a85cb22328647816280050b0440ad0698605d2dd30413a000e8abe627c6196f WatchSource:0}: Error finding container 4a85cb22328647816280050b0440ad0698605d2dd30413a000e8abe627c6196f: Status 404 returned error can't find the container with id 4a85cb22328647816280050b0440ad0698605d2dd30413a000e8abe627c6196f
Oct 31 21:33:58.793630 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:58.793592    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-371531-7b5cd97689-rtfrd" event={"ID":"9bcfc171-925d-4bfd-a2b6-61c8b90bc395","Type":"ContainerStarted","Data":"4a85cb22328647816280050b0440ad0698605d2dd30413a000e8abe627c6196f"}
Oct 31 21:33:58.952248 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:58.952154    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:33:59.039966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:59.032501    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 31 21:33:59.039966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:59.032567    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh" podUID="ffaf1375-c974-4e1d-a8bd-b880d7ea6e96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 31 21:33:59.054192 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:59.047545    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 31 21:33:59.054192 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:59.047604    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm" podUID="f9d9464c-85ec-4323-81b6-7201345e9f07" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 31 21:33:59.806641 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:59.806597    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-371531-7b5cd97689-rtfrd" event={"ID":"9bcfc171-925d-4bfd-a2b6-61c8b90bc395","Type":"ContainerStarted","Data":"3ed043e251d1c0964a5992ec138584f540def3f560856f241c7788053540e614"}
Oct 31 21:33:59.807230 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:59.806745    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-371531-7b5cd97689-rtfrd"
Oct 31 21:33:59.856512 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:33:59.856445    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-371531-7b5cd97689-rtfrd" podStartSLOduration=2.8564260519999998 podStartE2EDuration="2.856426052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:33:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:33:59.850663281 +0000 UTC m=+1079.766972712" watchObservedRunningTime="2026-10-31 21:33:59.856426052 +0000 UTC m=+1079.772735482"
Oct 31 21:34:00.060336 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:00.060252    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:34:00.061493 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:00.061472    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:34:00.068713 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:00.068681    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:34:00.069227 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:00.069208    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:34:00.071331 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:00.071315    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:34:00.074002 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:00.073983    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:34:00.650228 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:00.650192    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:02.470290 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:02.470257    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:04.412006 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:04.411956    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:06.302764 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:06.302719    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:08.011829 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:08.011791    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:09.902509 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:09.902470    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:10.662614 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:10.662564    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:10.822256 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:10.822224    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-371531-7b5cd97689-rtfrd"
Oct 31 21:34:11.790802 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:11.790769    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:11.953890 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:11.953854    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:12.065600 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:12.065509    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:13.611609 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:13.611567    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:14.926927 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:14.926891    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:34:15.015369 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:15.015334    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:15.425034 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:15.424996    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:15.645960 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:15.641550    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:15.753824 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:15.753741    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:16.155212 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.155179    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"]
Oct 31 21:34:16.161271 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.161244    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:34:16.181153 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.181118    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-248b29-openshift-default-dockercfg-q96ht\""
Oct 31 21:34:16.211635 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.211600    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"]
Oct 31 21:34:16.215851 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.215822    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:16.249488 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.249456    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istio-token\") pod \"e2e-api-b-248b29-openshift-default-66f64586d5-94rjj\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") " pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:34:16.249643 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.249499    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istiod-ca-cert\") pod \"e2e-api-b-248b29-openshift-default-66f64586d5-94rjj\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") " pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:34:16.249643 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.249571    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-workload-certs\") pod \"e2e-api-b-248b29-openshift-default-66f64586d5-94rjj\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") " pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:34:16.249749 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.249735    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-credential-socket\") pod \"e2e-api-b-248b29-openshift-default-66f64586d5-94rjj\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") " pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:34:16.249798 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.249757    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istio-podinfo\") pod \"e2e-api-b-248b29-openshift-default-66f64586d5-94rjj\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") " pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:34:16.249834 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.249796    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istio-envoy\") pod \"e2e-api-b-248b29-openshift-default-66f64586d5-94rjj\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") " pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:34:16.249879 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.249869    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-workload-socket\") pod \"e2e-api-b-248b29-openshift-default-66f64586d5-94rjj\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") " pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:34:16.249918 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.249912    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istio-data\") pod \"e2e-api-b-248b29-openshift-default-66f64586d5-94rjj\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") " pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:34:16.250053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.250039    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5qv7\" (UniqueName: \"kubernetes.io/projected/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-kube-api-access-b5qv7\") pod \"e2e-api-b-248b29-openshift-default-66f64586d5-94rjj\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") " pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:34:16.354579 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.354537    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-credential-socket\") pod \"e2e-api-b-248b29-openshift-default-66f64586d5-94rjj\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") " pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:34:16.355512 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.355487    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istio-podinfo\") pod \"e2e-api-b-248b29-openshift-default-66f64586d5-94rjj\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") " pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:34:16.355849 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.355828    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istio-envoy\") pod \"e2e-api-b-248b29-openshift-default-66f64586d5-94rjj\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") " pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:34:16.356012 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.355990    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-workload-socket\") pod \"e2e-api-b-248b29-openshift-default-66f64586d5-94rjj\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") " pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:34:16.356145 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.356127    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istio-data\") pod \"e2e-api-b-248b29-openshift-default-66f64586d5-94rjj\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") " pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:34:16.356351 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.356335    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5qv7\" (UniqueName: \"kubernetes.io/projected/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-kube-api-access-b5qv7\") pod \"e2e-api-b-248b29-openshift-default-66f64586d5-94rjj\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") " pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:34:16.356652 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.356627    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istio-token\") pod \"e2e-api-b-248b29-openshift-default-66f64586d5-94rjj\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") " pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:34:16.358101 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.358081    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istiod-ca-cert\") pod \"e2e-api-b-248b29-openshift-default-66f64586d5-94rjj\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") " pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:34:16.359123 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.359103    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-workload-certs\") pod \"e2e-api-b-248b29-openshift-default-66f64586d5-94rjj\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") " pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:34:16.371188 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.355258    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-credential-socket\") pod \"e2e-api-b-248b29-openshift-default-66f64586d5-94rjj\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") " pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:34:16.371428 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.356826    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-workload-socket\") pod \"e2e-api-b-248b29-openshift-default-66f64586d5-94rjj\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") " pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:34:16.371538 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.358974    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istiod-ca-cert\") pod \"e2e-api-b-248b29-openshift-default-66f64586d5-94rjj\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") " pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:34:16.371646 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.357348    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istio-data\") pod \"e2e-api-b-248b29-openshift-default-66f64586d5-94rjj\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") " pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:34:16.371745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.359524    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-workload-certs\") pod \"e2e-api-b-248b29-openshift-default-66f64586d5-94rjj\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") " pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:34:16.375725 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.375692    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istio-podinfo\") pod \"e2e-api-b-248b29-openshift-default-66f64586d5-94rjj\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") " pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:34:16.376185 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.376109    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istio-envoy\") pod \"e2e-api-b-248b29-openshift-default-66f64586d5-94rjj\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") " pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:34:16.410225 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.410148    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5qv7\" (UniqueName: \"kubernetes.io/projected/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-kube-api-access-b5qv7\") pod \"e2e-api-b-248b29-openshift-default-66f64586d5-94rjj\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") " pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:34:16.428971 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.414497    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istio-token\") pod \"e2e-api-b-248b29-openshift-default-66f64586d5-94rjj\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") " pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:34:16.505103 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.487524    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:34:16.812187 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.812147    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:16.942102 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.942072    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"]
Oct 31 21:34:16.942490 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:34:16.942466    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cfc07f9_3f4a_445d_9880_a7b25a95b3dd.slice/crio-1fdbf904118a0d0936980850e8f480391bd9fdbf436c8e25bf4775df932154e9 WatchSource:0}: Error finding container 1fdbf904118a0d0936980850e8f480391bd9fdbf436c8e25bf4775df932154e9: Status 404 returned error can't find the container with id 1fdbf904118a0d0936980850e8f480391bd9fdbf436c8e25bf4775df932154e9
Oct 31 21:34:16.945106 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.945077    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:34:16.945175 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.945140    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:34:16.945215 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:16.945189    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:34:17.029043 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:17.029001    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj" event={"ID":"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd","Type":"ContainerStarted","Data":"adbd0fb8e5ff7354007505510805f27a19ac4d6b95f04a8274bbeb79a5ecd254"}
Oct 31 21:34:17.029209 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:17.029068    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj" event={"ID":"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd","Type":"ContainerStarted","Data":"1fdbf904118a0d0936980850e8f480391bd9fdbf436c8e25bf4775df932154e9"}
Oct 31 21:34:17.088449 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:17.088336    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj" podStartSLOduration=1.088320359 podStartE2EDuration="1.088320359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:34:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:34:17.084131592 +0000 UTC m=+1097.000441023" watchObservedRunningTime="2026-10-31 21:34:17.088320359 +0000 UTC m=+1097.004629789"
Oct 31 21:34:17.489416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:17.489327    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:34:17.681728 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:17.681693    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:18.494425 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:18.494396    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:34:19.042253 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:19.042217    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:34:19.043107 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:19.043091    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:34:19.590494 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:19.590447    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:19.704246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:19.704210    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:21.275246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:21.275205    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:21.390149 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:21.390107    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:21.454728 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:34:21.454662    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:34:21.660009 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:21.564404    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-248b29-7c8fd468f5-fm992"]
Oct 31 21:34:21.769229 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:21.769169    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-248b29-7c8fd468f5-fm992"]
Oct 31 21:34:21.771558 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:21.771525    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-248b29-7c8fd468f5-fm992"
Oct 31 21:34:21.847630 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:21.841454    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-248b29\""
Oct 31 21:34:21.847999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:21.847972    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-248b29-dockercfg-85g7l\""
Oct 31 21:34:21.916421 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:21.916336    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x7wm\" (UniqueName: \"kubernetes.io/projected/a9280996-adf2-439c-934a-e8fa3aac2ef2-kube-api-access-8x7wm\") pod \"payload-processing-e2e-api-b-248b29-7c8fd468f5-fm992\" (UID: \"a9280996-adf2-439c-934a-e8fa3aac2ef2\") " pod="openshift-ingress/payload-processing-e2e-api-b-248b29-7c8fd468f5-fm992"
Oct 31 21:34:21.916421 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:21.916387    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9280996-adf2-439c-934a-e8fa3aac2ef2-plugins-config-volume\") pod \"payload-processing-e2e-api-b-248b29-7c8fd468f5-fm992\" (UID: \"a9280996-adf2-439c-934a-e8fa3aac2ef2\") " pod="openshift-ingress/payload-processing-e2e-api-b-248b29-7c8fd468f5-fm992"
Oct 31 21:34:22.018222 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:22.018190    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x7wm\" (UniqueName: \"kubernetes.io/projected/a9280996-adf2-439c-934a-e8fa3aac2ef2-kube-api-access-8x7wm\") pod \"payload-processing-e2e-api-b-248b29-7c8fd468f5-fm992\" (UID: \"a9280996-adf2-439c-934a-e8fa3aac2ef2\") " pod="openshift-ingress/payload-processing-e2e-api-b-248b29-7c8fd468f5-fm992"
Oct 31 21:34:22.018403 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:22.018245    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9280996-adf2-439c-934a-e8fa3aac2ef2-plugins-config-volume\") pod \"payload-processing-e2e-api-b-248b29-7c8fd468f5-fm992\" (UID: \"a9280996-adf2-439c-934a-e8fa3aac2ef2\") " pod="openshift-ingress/payload-processing-e2e-api-b-248b29-7c8fd468f5-fm992"
Oct 31 21:34:22.018994 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:22.018971    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9280996-adf2-439c-934a-e8fa3aac2ef2-plugins-config-volume\") pod \"payload-processing-e2e-api-b-248b29-7c8fd468f5-fm992\" (UID: \"a9280996-adf2-439c-934a-e8fa3aac2ef2\") " pod="openshift-ingress/payload-processing-e2e-api-b-248b29-7c8fd468f5-fm992"
Oct 31 21:34:22.027835 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:22.027807    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:22.042855 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:22.042826    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x7wm\" (UniqueName: \"kubernetes.io/projected/a9280996-adf2-439c-934a-e8fa3aac2ef2-kube-api-access-8x7wm\") pod \"payload-processing-e2e-api-b-248b29-7c8fd468f5-fm992\" (UID: \"a9280996-adf2-439c-934a-e8fa3aac2ef2\") " pod="openshift-ingress/payload-processing-e2e-api-b-248b29-7c8fd468f5-fm992"
Oct 31 21:34:22.158254 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:22.158220    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-248b29-7c8fd468f5-fm992"
Oct 31 21:34:22.486871 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:22.486775    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-248b29-7c8fd468f5-fm992"]
Oct 31 21:34:22.510619 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:34:22.510581    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9280996_adf2_439c_934a_e8fa3aac2ef2.slice/crio-cd78a7051bee3f34ac0e7006fe6f53f605402d8a730ff3b25048e844c3570adb WatchSource:0}: Error finding container cd78a7051bee3f34ac0e7006fe6f53f605402d8a730ff3b25048e844c3570adb: Status 404 returned error can't find the container with id cd78a7051bee3f34ac0e7006fe6f53f605402d8a730ff3b25048e844c3570adb
Oct 31 21:34:23.071125 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:23.071090    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-248b29-7c8fd468f5-fm992" event={"ID":"a9280996-adf2-439c-934a-e8fa3aac2ef2","Type":"ContainerStarted","Data":"7773354f12bb59f9635d17d24402abb89d6306670563ee788362db1ee9ea127c"}
Oct 31 21:34:23.071303 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:23.071141    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-248b29-7c8fd468f5-fm992" event={"ID":"a9280996-adf2-439c-934a-e8fa3aac2ef2","Type":"ContainerStarted","Data":"cd78a7051bee3f34ac0e7006fe6f53f605402d8a730ff3b25048e844c3570adb"}
Oct 31 21:34:23.071371 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:23.071303    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-248b29-7c8fd468f5-fm992"
Oct 31 21:34:23.117072 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:23.117023    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-248b29-7c8fd468f5-fm992" podStartSLOduration=2.117011186 podStartE2EDuration="2.117011186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:34:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:34:23.11468341 +0000 UTC m=+1103.030992841" watchObservedRunningTime="2026-10-31 21:34:23.117011186 +0000 UTC m=+1103.033320617"
Oct 31 21:34:23.512574 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:23.512409    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:23.972152 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:23.972058    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:25.719827 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:25.719781    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:27.421853 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:27.421816    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:29.153915 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:29.153872    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:30.922742 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:30.922692    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:32.905513 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:32.905466    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:34.103301 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:34.103270    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-248b29-7c8fd468f5-fm992"
Oct 31 21:34:34.708054 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:34.708022    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:35.730415 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:35.730380    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:36.314519 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:36.314463    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:38.888626 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:38.888581    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:39.014064 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:39.014025    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:39.290893 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:39.290855    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:40.010615 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:40.010564    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:34:40.012534 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:40.012499    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n" podUID="4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:34:40.240092 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:40.240041    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt" podUID="9771611f-879a-44f6-9c1a-f8647dc2d841" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:34:40.524177 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:40.524127    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh" podUID="869ed893-352c-4f8f-a174-b42afdeb4de7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.77:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:34:40.752012 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:40.736152    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" podUID="af7e0e0c-cf52-4270-b528-35718339996c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Oct 31 21:34:40.913372 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:40.912230    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podUID="9cd67bf5-8833-4a82-bc33-bed7bdfd4d36" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:34:41.135966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:41.111143    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n" podUID="4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": context deadline exceeded"
Oct 31 21:34:41.136967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:41.136695    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:34:41.241185 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:41.241067    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt" podUID="9771611f-879a-44f6-9c1a-f8647dc2d841" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:34:41.355855 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:41.355820    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:34:41.549021 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:41.517911    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:34:41.651964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:41.650079    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:34:41.712844 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:41.712809    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:43.441777 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:43.441673    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:45.123863 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:45.123820    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:45.720087 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:45.720043    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:46.870876 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:46.870843    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:47.502342 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:47.502301    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-90db52-57478fddb7-vq79w"]
Oct 31 21:34:47.548555 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:47.548489    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-90db52-57f7c9d8dc-ltfg6"]
Oct 31 21:34:47.667879 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:47.667847    4460 generic.go:356] "Generic (PLEG): container finished" podID="8842af32-490d-40d9-9da3-e9daba253252" containerID="939d3519c52c12e7c628467a6e348dbbfd3ec477c78d265f11274b60d5cc3919" exitCode=0
Oct 31 21:34:47.668063 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:47.667895    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr" event={"ID":"8842af32-490d-40d9-9da3-e9daba253252","Type":"ContainerDied","Data":"939d3519c52c12e7c628467a6e348dbbfd3ec477c78d265f11274b60d5cc3919"}
Oct 31 21:34:47.668432 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:47.668411    4460 scope.go:122] "RemoveContainer" containerID="939d3519c52c12e7c628467a6e348dbbfd3ec477c78d265f11274b60d5cc3919"
Oct 31 21:34:47.670232 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:47.670201    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:34:47.670334 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:47.670291    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:34:47.670390 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:47.670338    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:34:47.677230 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:47.673278    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:47.789277 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:47.789240    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:48.677016 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:48.676975    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr" event={"ID":"8842af32-490d-40d9-9da3-e9daba253252","Type":"ContainerStarted","Data":"19d115c4f1afe5fa2f1c4cdd1b0fa3605dcf3a98e5fefc7dd058be1f05bb1eee"}
Oct 31 21:34:48.745983 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:48.744396    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:49.236510 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:49.232357    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:34:49.236975 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:49.236913    4460 patch_prober.go:29] interesting pod/e2e-api-b-371531-openshift-default-d977d868c-nzsxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 31 21:34:49.237172 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:49.237143    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr" podUID="8842af32-490d-40d9-9da3-e9daba253252" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 31 21:34:50.245837 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:50.245324    4460 patch_prober.go:29] interesting pod/e2e-api-b-371531-openshift-default-d977d868c-nzsxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 31 21:34:50.245837 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:50.245393    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr" podUID="8842af32-490d-40d9-9da3-e9daba253252" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 31 21:34:50.308530 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:50.308493    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:51.245129 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:51.245086    4460 patch_prober.go:29] interesting pod/e2e-api-b-371531-openshift-default-d977d868c-nzsxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 31 21:34:51.245348 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:51.245167    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr" podUID="8842af32-490d-40d9-9da3-e9daba253252" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 31 21:34:51.474137 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:34:51.474096    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:34:51.876285 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:51.876240    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:52.242031 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:52.241440    4460 patch_prober.go:29] interesting pod/e2e-api-b-371531-openshift-default-d977d868c-nzsxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 31 21:34:52.242031 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:52.241523    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr" podUID="8842af32-490d-40d9-9da3-e9daba253252" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 31 21:34:52.503763 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:52.503659    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-90db52-57478fddb7-vq79w" podUID="086c327e-0e66-4f9c-83fb-85f688ffa253" containerName="payload-processing" containerID="cri-o://8a4e597c08993bf07073ad5a1ad2a89ed76c6e064b1488737c30d545f4156a78" gracePeriod=25
Oct 31 21:34:52.549419 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:52.549372    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-90db52-57f7c9d8dc-ltfg6" podUID="9afe6453-662b-4786-8f53-b9d6d720750a" containerName="payload-pre-processing" containerID="cri-o://5be6111b0c923c5236b266b2bbdd8f14068af20fe2c84aed8f5de0c87d1af5cf" gracePeriod=25
Oct 31 21:34:52.711185 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:52.711142    4460 generic.go:356] "Generic (PLEG): container finished" podID="086c327e-0e66-4f9c-83fb-85f688ffa253" containerID="8a4e597c08993bf07073ad5a1ad2a89ed76c6e064b1488737c30d545f4156a78" exitCode=0
Oct 31 21:34:52.711362 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:52.711192    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-90db52-57478fddb7-vq79w" event={"ID":"086c327e-0e66-4f9c-83fb-85f688ffa253","Type":"ContainerDied","Data":"8a4e597c08993bf07073ad5a1ad2a89ed76c6e064b1488737c30d545f4156a78"}
Oct 31 21:34:52.713052 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:52.713025    4460 generic.go:356] "Generic (PLEG): container finished" podID="9afe6453-662b-4786-8f53-b9d6d720750a" containerID="5be6111b0c923c5236b266b2bbdd8f14068af20fe2c84aed8f5de0c87d1af5cf" exitCode=0
Oct 31 21:34:52.713197 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:52.713096    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-90db52-57f7c9d8dc-ltfg6" event={"ID":"9afe6453-662b-4786-8f53-b9d6d720750a","Type":"ContainerDied","Data":"5be6111b0c923c5236b266b2bbdd8f14068af20fe2c84aed8f5de0c87d1af5cf"}
Oct 31 21:34:52.895385 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:52.895355    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-90db52-57478fddb7-vq79w"
Oct 31 21:34:52.935967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:52.919639    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-90db52-57f7c9d8dc-ltfg6"
Oct 31 21:34:52.988880 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:52.988837    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/086c327e-0e66-4f9c-83fb-85f688ffa253-kube-api-access-wn2kh\" (UniqueName: \"kubernetes.io/projected/086c327e-0e66-4f9c-83fb-85f688ffa253-kube-api-access-wn2kh\") pod \"086c327e-0e66-4f9c-83fb-85f688ffa253\" (UID: \"086c327e-0e66-4f9c-83fb-85f688ffa253\") "
Oct 31 21:34:52.989105 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:52.988924    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/086c327e-0e66-4f9c-83fb-85f688ffa253-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/086c327e-0e66-4f9c-83fb-85f688ffa253-plugins-config-volume\") pod \"086c327e-0e66-4f9c-83fb-85f688ffa253\" (UID: \"086c327e-0e66-4f9c-83fb-85f688ffa253\") "
Oct 31 21:34:52.990032 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:52.989997    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/086c327e-0e66-4f9c-83fb-85f688ffa253-plugins-config-volume" pod "086c327e-0e66-4f9c-83fb-85f688ffa253" (UID: "086c327e-0e66-4f9c-83fb-85f688ffa253"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:34:52.992073 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:52.992042    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/086c327e-0e66-4f9c-83fb-85f688ffa253-kube-api-access-wn2kh" pod "086c327e-0e66-4f9c-83fb-85f688ffa253" (UID: "086c327e-0e66-4f9c-83fb-85f688ffa253"). InnerVolumeSpecName "kube-api-access-wn2kh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:34:53.091098 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:53.090999    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9afe6453-662b-4786-8f53-b9d6d720750a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9afe6453-662b-4786-8f53-b9d6d720750a-plugins-config-volume\") pod \"9afe6453-662b-4786-8f53-b9d6d720750a\" (UID: \"9afe6453-662b-4786-8f53-b9d6d720750a\") "
Oct 31 21:34:53.091287 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:53.091268    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9afe6453-662b-4786-8f53-b9d6d720750a-kube-api-access-sh95j\" (UniqueName: \"kubernetes.io/projected/9afe6453-662b-4786-8f53-b9d6d720750a-kube-api-access-sh95j\") pod \"9afe6453-662b-4786-8f53-b9d6d720750a\" (UID: \"9afe6453-662b-4786-8f53-b9d6d720750a\") "
Oct 31 21:34:53.091404 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:53.091380    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9afe6453-662b-4786-8f53-b9d6d720750a-plugins-config-volume" pod "9afe6453-662b-4786-8f53-b9d6d720750a" (UID: "9afe6453-662b-4786-8f53-b9d6d720750a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:34:53.091846 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:53.091820    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wn2kh\" (UniqueName: \"kubernetes.io/projected/086c327e-0e66-4f9c-83fb-85f688ffa253-kube-api-access-wn2kh\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:34:53.091846 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:53.091844    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9afe6453-662b-4786-8f53-b9d6d720750a-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:34:53.091846 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:53.091855    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/086c327e-0e66-4f9c-83fb-85f688ffa253-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:34:53.093575 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:53.093544    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9afe6453-662b-4786-8f53-b9d6d720750a-kube-api-access-sh95j" pod "9afe6453-662b-4786-8f53-b9d6d720750a" (UID: "9afe6453-662b-4786-8f53-b9d6d720750a"). InnerVolumeSpecName "kube-api-access-sh95j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:34:53.193453 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:53.193419    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sh95j\" (UniqueName: \"kubernetes.io/projected/9afe6453-662b-4786-8f53-b9d6d720750a-kube-api-access-sh95j\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:34:53.240966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:53.236426    4460 patch_prober.go:29] interesting pod/e2e-api-b-371531-openshift-default-d977d868c-nzsxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 31 21:34:53.240966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:53.236500    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr" podUID="8842af32-490d-40d9-9da3-e9daba253252" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 31 21:34:53.547825 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:53.547770    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:53.693966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:53.690126    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:34:53.777961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:53.769576    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-90db52-57478fddb7-vq79w" event={"ID":"086c327e-0e66-4f9c-83fb-85f688ffa253","Type":"ContainerDied","Data":"2e9968b15b6440ce6a6b0876f518b448017c5ddb461be3b6f815204b3d5ef81e"}
Oct 31 21:34:53.777961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:53.769653    4460 scope.go:122] "RemoveContainer" containerID="8a4e597c08993bf07073ad5a1ad2a89ed76c6e064b1488737c30d545f4156a78"
Oct 31 21:34:53.777961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:53.769808    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-90db52-57478fddb7-vq79w"
Oct 31 21:34:53.777961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:53.775611    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-90db52-57f7c9d8dc-ltfg6" event={"ID":"9afe6453-662b-4786-8f53-b9d6d720750a","Type":"ContainerDied","Data":"60a4517cb5a5a8a83be354c48847cefdb3b0129a35a2fe5d5675e9d249055375"}
Oct 31 21:34:53.777961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:53.775760    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-90db52-57f7c9d8dc-ltfg6"
Oct 31 21:34:53.817796 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:53.810369    4460 scope.go:122] "RemoveContainer" containerID="5be6111b0c923c5236b266b2bbdd8f14068af20fe2c84aed8f5de0c87d1af5cf"
Oct 31 21:34:53.871612 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:53.871579    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-90db52-57478fddb7-vq79w"]
Oct 31 21:34:53.884970 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:53.884916    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-90db52-57478fddb7-vq79w"]
Oct 31 21:34:53.912406 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:53.912362    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-90db52-57f7c9d8dc-ltfg6"]
Oct 31 21:34:53.918629 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:53.918597    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-90db52-57f7c9d8dc-ltfg6"]
Oct 31 21:34:54.228186 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.228083    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="086c327e-0e66-4f9c-83fb-85f688ffa253" path="/var/lib/kubelet/pods/086c327e-0e66-4f9c-83fb-85f688ffa253/volumes"
Oct 31 21:34:54.228983 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.228957    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9afe6453-662b-4786-8f53-b9d6d720750a" path="/var/lib/kubelet/pods/9afe6453-662b-4786-8f53-b9d6d720750a/volumes"
Oct 31 21:34:54.249352 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.249285    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:34:54.251079 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.251051    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:34:54.357430 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.357393    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:54.757747 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.757709    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"]
Oct 31 21:34:54.758828 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.758799    4460 cpu_manager.go:414] "containerMap: removing container" podUID="086c327e-0e66-4f9c-83fb-85f688ffa253" containerName="payload-processing"
Oct 31 21:34:54.758828 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.758827    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="086c327e-0e66-4f9c-83fb-85f688ffa253" containerName="payload-processing"
Oct 31 21:34:54.758998 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.758852    4460 cpu_manager.go:414] "containerMap: removing container" podUID="9afe6453-662b-4786-8f53-b9d6d720750a" containerName="payload-pre-processing"
Oct 31 21:34:54.758998 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.758860    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9afe6453-662b-4786-8f53-b9d6d720750a" containerName="payload-pre-processing"
Oct 31 21:34:54.759095 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.759077    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="086c327e-0e66-4f9c-83fb-85f688ffa253" containerName="payload-processing"
Oct 31 21:34:54.759130 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.759093    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="9afe6453-662b-4786-8f53-b9d6d720750a" containerName="payload-pre-processing"
Oct 31 21:34:54.764701 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.764677    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:34:54.784844 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.784816    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"]
Oct 31 21:34:54.793148 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.793117    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-08ddf0-openshift-default-dockercfg-p8r86\""
Oct 31 21:34:54.804314 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.804280    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:54.821045 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.820999    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-workload-certs\") pod \"e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") " pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:34:54.821045 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.821049    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istiod-ca-cert\") pod \"e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") " pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:34:54.821324 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.821106    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istio-token\") pod \"e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") " pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:34:54.821324 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.821268    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd8cc\" (UniqueName: \"kubernetes.io/projected/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-kube-api-access-pd8cc\") pod \"e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") " pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:34:54.821475 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.821450    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istio-podinfo\") pod \"e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") " pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:34:54.821538 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.821513    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-workload-socket\") pod \"e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") " pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:34:54.821594 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.821548    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istio-data\") pod \"e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") " pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:34:54.821653 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.821592    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istio-envoy\") pod \"e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") " pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:34:54.821748 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.821726    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-credential-socket\") pod \"e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") " pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:34:54.929963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.923292    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-credential-socket\") pod \"e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") " pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:34:54.929963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.923424    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-workload-certs\") pod \"e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") " pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:34:54.929963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.923474    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istiod-ca-cert\") pod \"e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") " pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:34:54.929963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.923514    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istio-token\") pod \"e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") " pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:34:54.929963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.923650    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd8cc\" (UniqueName: \"kubernetes.io/projected/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-kube-api-access-pd8cc\") pod \"e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") " pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:34:54.929963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.923737    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istio-podinfo\") pod \"e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") " pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:34:54.929963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.923783    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-workload-socket\") pod \"e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") " pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:34:54.929963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.923815    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istio-data\") pod \"e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") " pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:34:54.929963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.923845    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istio-envoy\") pod \"e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") " pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:34:54.945964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.940453    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-credential-socket\") pod \"e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") " pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:34:54.961991 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.954275    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-workload-socket\") pod \"e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") " pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:34:54.961991 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.954842    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-workload-certs\") pod \"e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") " pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:34:54.961991 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.955106    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istio-data\") pod \"e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") " pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:34:54.961991 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.956725    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istiod-ca-cert\") pod \"e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") " pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:34:54.988977 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.988914    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istio-envoy\") pod \"e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") " pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:34:55.013015 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:54.994874    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istio-podinfo\") pod \"e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") " pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:34:55.013015 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:55.011128    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd8cc\" (UniqueName: \"kubernetes.io/projected/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-kube-api-access-pd8cc\") pod \"e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") " pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:34:55.057962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:55.056389    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istio-token\") pod \"e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") " pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:34:55.095019 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:55.090317    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:34:55.342542 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:55.342441    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:55.522519 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:55.522488    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"]
Oct 31 21:34:55.524034 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:34:55.523994    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod485bbdc1_0d53_47b1_9b33_1b89cbf7f5c5.slice/crio-d00e3f57549a92f3376469df7d6c9cb94159c1cf19037b838f89fe63f0ddaf4e WatchSource:0}: Error finding container d00e3f57549a92f3376469df7d6c9cb94159c1cf19037b838f89fe63f0ddaf4e: Status 404 returned error can't find the container with id d00e3f57549a92f3376469df7d6c9cb94159c1cf19037b838f89fe63f0ddaf4e
Oct 31 21:34:55.526894 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:55.526862    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:34:55.526987 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:55.526933    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:34:55.527034 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:55.527008    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:34:55.545909 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:55.545878    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:55.799880 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:55.799836    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z" event={"ID":"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5","Type":"ContainerStarted","Data":"10cbadfd2d786bbb06634b3e2368ad837f3385778fe526b74cc998e38dcfc6c4"}
Oct 31 21:34:55.800389 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:55.799909    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z" event={"ID":"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5","Type":"ContainerStarted","Data":"d00e3f57549a92f3376469df7d6c9cb94159c1cf19037b838f89fe63f0ddaf4e"}
Oct 31 21:34:55.849740 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:55.849620    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z" podStartSLOduration=1.849604749 podStartE2EDuration="1.849604749s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:34:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:34:55.848164227 +0000 UTC m=+1135.764473659" watchObservedRunningTime="2026-10-31 21:34:55.849604749 +0000 UTC m=+1135.765914253"
Oct 31 21:34:56.091757 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:56.091035    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:34:56.113631 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:56.113594    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:34:56.807371 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:56.807340    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:34:56.808538 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:56.808517    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:34:57.317662 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:57.317619    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:58.198039 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:58.197997    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:58.320377 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:58.320337    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:59.122776 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:59.122736    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:34:59.340637 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:34:59.340597    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:35:00.091963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:00.091320    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:00.259634 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:00.259570    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:00.290643 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:00.290610    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-08ddf0-54474ff8d7-vxf9n"]
Oct 31 21:35:00.315112 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:00.315071    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-08ddf0-54474ff8d7-vxf9n"]
Oct 31 21:35:00.315545 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:00.315495    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-08ddf0-54474ff8d7-vxf9n"
Oct 31 21:35:00.334708 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:00.334668    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-08ddf0\""
Oct 31 21:35:00.502971 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:00.449290    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ca7b94b-2aef-45f5-8551-7ebfd685fe4b-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-08ddf0-54474ff8d7-vxf9n\" (UID: \"4ca7b94b-2aef-45f5-8551-7ebfd685fe4b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-08ddf0-54474ff8d7-vxf9n"
Oct 31 21:35:00.621011 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:00.605197    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q25n4\" (UniqueName: \"kubernetes.io/projected/4ca7b94b-2aef-45f5-8551-7ebfd685fe4b-kube-api-access-q25n4\") pod \"payload-pre-processing-e2e-api-a-08ddf0-54474ff8d7-vxf9n\" (UID: \"4ca7b94b-2aef-45f5-8551-7ebfd685fe4b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-08ddf0-54474ff8d7-vxf9n"
Oct 31 21:35:00.707962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:00.707902    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q25n4\" (UniqueName: \"kubernetes.io/projected/4ca7b94b-2aef-45f5-8551-7ebfd685fe4b-kube-api-access-q25n4\") pod \"payload-pre-processing-e2e-api-a-08ddf0-54474ff8d7-vxf9n\" (UID: \"4ca7b94b-2aef-45f5-8551-7ebfd685fe4b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-08ddf0-54474ff8d7-vxf9n"
Oct 31 21:35:00.708529 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:00.708499    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ca7b94b-2aef-45f5-8551-7ebfd685fe4b-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-08ddf0-54474ff8d7-vxf9n\" (UID: \"4ca7b94b-2aef-45f5-8551-7ebfd685fe4b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-08ddf0-54474ff8d7-vxf9n"
Oct 31 21:35:00.709915 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:00.709838    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ca7b94b-2aef-45f5-8551-7ebfd685fe4b-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-08ddf0-54474ff8d7-vxf9n\" (UID: \"4ca7b94b-2aef-45f5-8551-7ebfd685fe4b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-08ddf0-54474ff8d7-vxf9n"
Oct 31 21:35:00.735994 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:00.735850    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q25n4\" (UniqueName: \"kubernetes.io/projected/4ca7b94b-2aef-45f5-8551-7ebfd685fe4b-kube-api-access-q25n4\") pod \"payload-pre-processing-e2e-api-a-08ddf0-54474ff8d7-vxf9n\" (UID: \"4ca7b94b-2aef-45f5-8551-7ebfd685fe4b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-08ddf0-54474ff8d7-vxf9n"
Oct 31 21:35:00.940966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:00.940350    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-08ddf0-54474ff8d7-vxf9n"
Oct 31 21:35:01.383726 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:01.367229    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:01.568597 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:01.568368    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-08ddf0-54474ff8d7-vxf9n"]
Oct 31 21:35:01.568922 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:35:01.568657    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ca7b94b_2aef_45f5_8551_7ebfd685fe4b.slice/crio-fa2cf33d30517994e98777f73afc455b328425269ee4c2a50237df0934f34846 WatchSource:0}: Error finding container fa2cf33d30517994e98777f73afc455b328425269ee4c2a50237df0934f34846: Status 404 returned error can't find the container with id fa2cf33d30517994e98777f73afc455b328425269ee4c2a50237df0934f34846
Oct 31 21:35:01.613311 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:01.613280    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:01.738963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:01.738718    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:01.870134 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:01.868764    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-08ddf0-54474ff8d7-vxf9n" event={"ID":"4ca7b94b-2aef-45f5-8551-7ebfd685fe4b","Type":"ContainerStarted","Data":"4b7c86eaef4b71dcefbe923a2395f6bf2b650dc564c5a4ff9f861d15ff0dd689"}
Oct 31 21:35:01.870134 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:01.868841    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-08ddf0-54474ff8d7-vxf9n" event={"ID":"4ca7b94b-2aef-45f5-8551-7ebfd685fe4b","Type":"ContainerStarted","Data":"fa2cf33d30517994e98777f73afc455b328425269ee4c2a50237df0934f34846"}
Oct 31 21:35:01.870134 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:01.870059    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-08ddf0-54474ff8d7-vxf9n"
Oct 31 21:35:01.952024 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:01.949658    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-08ddf0-54474ff8d7-vxf9n" podStartSLOduration=1.949635479 podStartE2EDuration="1.949635479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:35:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:35:01.930985835 +0000 UTC m=+1141.847295268" watchObservedRunningTime="2026-10-31 21:35:01.949635479 +0000 UTC m=+1141.865944910"
Oct 31 21:35:03.213503 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:03.213462    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:04.941934 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:04.941900    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:05.213197 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:05.213106    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:06.757076 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:06.757039    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:08.431107 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:08.431070    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:10.274233 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:10.274195    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:10.948082 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:10.948045    4460 generic.go:356] "Generic (PLEG): container finished" podID="8cfc07f9-3f4a-445d-9880-a7b25a95b3dd" containerID="adbd0fb8e5ff7354007505510805f27a19ac4d6b95f04a8274bbeb79a5ecd254" exitCode=0
Oct 31 21:35:10.948293 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:10.948115    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj" event={"ID":"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd","Type":"ContainerDied","Data":"adbd0fb8e5ff7354007505510805f27a19ac4d6b95f04a8274bbeb79a5ecd254"}
Oct 31 21:35:10.948585 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:10.948571    4460 scope.go:122] "RemoveContainer" containerID="adbd0fb8e5ff7354007505510805f27a19ac4d6b95f04a8274bbeb79a5ecd254"
Oct 31 21:35:10.950059 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:10.950027    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:35:10.950117 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:10.950104    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:35:10.950152 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:10.950134    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:35:11.774851 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:11.774813    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:11.886965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:11.886065    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:11.982156 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:11.981220    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj" event={"ID":"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd","Type":"ContainerStarted","Data":"216125795b7ad5da329a61eccb02690b327e33a376ff35edca312388f6fe88f4"}
Oct 31 21:35:12.489020 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:12.488989    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:35:12.489246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:12.489227    4460 patch_prober.go:29] interesting pod/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 31 21:35:12.489305 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:12.489271    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj" podUID="8cfc07f9-3f4a-445d-9880-a7b25a95b3dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 31 21:35:13.489657 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:13.489623    4460 patch_prober.go:29] interesting pod/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 31 21:35:13.490076 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:13.489673    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj" podUID="8cfc07f9-3f4a-445d-9880-a7b25a95b3dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 31 21:35:13.545790 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:13.545745    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:13.897200 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:13.897168    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-08ddf0-54474ff8d7-vxf9n"
Oct 31 21:35:14.489140 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:14.489106    4460 patch_prober.go:29] interesting pod/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 31 21:35:14.489307 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:14.489175    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj" podUID="8cfc07f9-3f4a-445d-9880-a7b25a95b3dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 31 21:35:15.095405 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:15.095370    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k"]
Oct 31 21:35:15.098785 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:15.098766    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k"
Oct 31 21:35:15.141965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:15.130180    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-08ddf0-serving-cert\""
Oct 31 21:35:15.152088 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:15.152056    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k"]
Oct 31 21:35:15.181055 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:15.181016    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:15.252082 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:15.252046    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8-maas-api-tls\") pod \"maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k\" (UID: \"6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k"
Oct 31 21:35:15.252313 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:15.252173    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpl69\" (UniqueName: \"kubernetes.io/projected/6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8-kube-api-access-rpl69\") pod \"maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k\" (UID: \"6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k"
Oct 31 21:35:15.252412 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:15.252394    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8-openshift-service-ca\") pod \"maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k\" (UID: \"6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k"
Oct 31 21:35:15.252449 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:15.252424    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8-metrics-certs\") pod \"maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k\" (UID: \"6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k"
Oct 31 21:35:15.360389 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:15.360293    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8-openshift-service-ca\") pod \"maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k\" (UID: \"6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k"
Oct 31 21:35:15.362858 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:15.362827    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8-metrics-certs\") pod \"maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k\" (UID: \"6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k"
Oct 31 21:35:15.363758 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:15.363733    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8-maas-api-tls\") pod \"maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k\" (UID: \"6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k"
Oct 31 21:35:15.364908 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:15.364888    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpl69\" (UniqueName: \"kubernetes.io/projected/6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8-kube-api-access-rpl69\") pod \"maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k\" (UID: \"6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k"
Oct 31 21:35:15.365802 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:15.361562    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8-openshift-service-ca\") pod \"maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k\" (UID: \"6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k"
Oct 31 21:35:15.387966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:15.385539    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8-metrics-certs\") pod \"maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k\" (UID: \"6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k"
Oct 31 21:35:15.405791 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:15.405757    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8-maas-api-tls\") pod \"maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k\" (UID: \"6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k"
Oct 31 21:35:15.452962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:15.416148    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpl69\" (UniqueName: \"kubernetes.io/projected/6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8-kube-api-access-rpl69\") pod \"maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k\" (UID: \"6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k"
Oct 31 21:35:15.469963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:15.460195    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:15.489518 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:15.489482    4460 patch_prober.go:29] interesting pod/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 31 21:35:15.489881 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:15.489839    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj" podUID="8cfc07f9-3f4a-445d-9880-a7b25a95b3dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 31 21:35:15.569638 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:15.569602    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:15.711867 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:15.711786    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k"
Oct 31 21:35:16.439240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:16.439207    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k"]
Oct 31 21:35:16.496966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:16.494615    4460 patch_prober.go:29] interesting pod/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 31 21:35:16.496966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:16.494684    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj" podUID="8cfc07f9-3f4a-445d-9880-a7b25a95b3dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 31 21:35:17.112355 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:17.112305    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:35:17.450432 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:17.450390    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k" event={"ID":"6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8","Type":"ContainerStarted","Data":"4fe2c9c5de72aafbc8dce1b4c9ff5844f9bac3f22e9386777c5ac5b8b7145ef2"}
Oct 31 21:35:17.499046 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:17.499010    4460 patch_prober.go:29] interesting pod/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Oct 31 21:35:17.499296 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:17.499270    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj" podUID="8cfc07f9-3f4a-445d-9880-a7b25a95b3dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Oct 31 21:35:17.688328 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:17.656102    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-hcd5s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 21:35:17.688328 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:17.656204    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s" podUID="c0272849-0715-4e09-8d0b-827116fb35d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:35:17.690147 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:17.690075    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-94gsm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 21:35:17.713175 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:17.713129    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm" podUID="ae31c154-bb41-4ff0-aae3-93164dc757df" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:35:17.791161 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:17.791110    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" podUID="af7e0e0c-cf52-4270-b528-35718339996c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:35:18.044244 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:18.037883    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:18.470922 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:18.470827    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k" event={"ID":"6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8","Type":"ContainerStarted","Data":"bb74edf28465cc677ed9989d10111ea68f9f67fada1ecb18bf09667d3a5ecd47"}
Oct 31 21:35:18.472134 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:18.472110    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k"
Oct 31 21:35:18.497813 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:18.497776    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:35:18.498411 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:18.498377    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:35:18.499373 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:18.499340    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:35:18.541029 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:18.537983    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k" podStartSLOduration=2.8490942219999997 podStartE2EDuration="3.537964949s" totalImagesPullingTime="688.870727ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:35:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:35:18.519899075 +0000 UTC m=+1158.436208520" watchObservedRunningTime="2026-10-31 21:35:18.537964949 +0000 UTC m=+1158.454274379"
Oct 31 21:35:18.648966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:18.647987    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7ac3c8-76bc49454f-vfdpw"]
Oct 31 21:35:18.793010 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:18.792916    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:18.914958 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:18.914896    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:20.408855 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:20.408795    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:21.454554 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:35:21.454511    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:35:22.329292 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:22.329249    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:23.648722 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:23.648666    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-7ac3c8-76bc49454f-vfdpw" podUID="8497c5c1-966e-48c3-beef-d982c8ab7627" containerName="payload-processing" containerID="cri-o://ad8fea4da5a956400f0441e12c1dd1bcd7d1c33fb7d78620b3ca63bb0183024f" gracePeriod=25
Oct 31 21:35:23.928579 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:23.928552    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-7ac3c8-76bc49454f-vfdpw"
Oct 31 21:35:23.979983 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:23.979952    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:24.156898 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:24.156816    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8497c5c1-966e-48c3-beef-d982c8ab7627-kube-api-access-rxrw8\" (UniqueName: \"kubernetes.io/projected/8497c5c1-966e-48c3-beef-d982c8ab7627-kube-api-access-rxrw8\") pod \"8497c5c1-966e-48c3-beef-d982c8ab7627\" (UID: \"8497c5c1-966e-48c3-beef-d982c8ab7627\") "
Oct 31 21:35:24.156898 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:24.156872    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8497c5c1-966e-48c3-beef-d982c8ab7627-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8497c5c1-966e-48c3-beef-d982c8ab7627-plugins-config-volume\") pod \"8497c5c1-966e-48c3-beef-d982c8ab7627\" (UID: \"8497c5c1-966e-48c3-beef-d982c8ab7627\") "
Oct 31 21:35:24.157347 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:24.157318    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8497c5c1-966e-48c3-beef-d982c8ab7627-plugins-config-volume" pod "8497c5c1-966e-48c3-beef-d982c8ab7627" (UID: "8497c5c1-966e-48c3-beef-d982c8ab7627"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:35:24.157774 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:24.157751    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8497c5c1-966e-48c3-beef-d982c8ab7627-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:35:24.159248 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:24.159220    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8497c5c1-966e-48c3-beef-d982c8ab7627-kube-api-access-rxrw8" pod "8497c5c1-966e-48c3-beef-d982c8ab7627" (UID: "8497c5c1-966e-48c3-beef-d982c8ab7627"). InnerVolumeSpecName "kube-api-access-rxrw8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:35:24.259609 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:24.259571    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxrw8\" (UniqueName: \"kubernetes.io/projected/8497c5c1-966e-48c3-beef-d982c8ab7627-kube-api-access-rxrw8\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:35:24.512895 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:24.512870    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k"
Oct 31 21:35:24.576886 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:24.576844    4460 generic.go:356] "Generic (PLEG): container finished" podID="8497c5c1-966e-48c3-beef-d982c8ab7627" containerID="ad8fea4da5a956400f0441e12c1dd1bcd7d1c33fb7d78620b3ca63bb0183024f" exitCode=0
Oct 31 21:35:24.577135 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:24.577109    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-7ac3c8-76bc49454f-vfdpw" event={"ID":"8497c5c1-966e-48c3-beef-d982c8ab7627","Type":"ContainerDied","Data":"ad8fea4da5a956400f0441e12c1dd1bcd7d1c33fb7d78620b3ca63bb0183024f"}
Oct 31 21:35:24.577215 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:24.577176    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-7ac3c8-76bc49454f-vfdpw" event={"ID":"8497c5c1-966e-48c3-beef-d982c8ab7627","Type":"ContainerDied","Data":"95ad3b429208d154c5ecc1f5e4854dfc871c061828b8493825acf8ec91c546fa"}
Oct 31 21:35:24.577215 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:24.577210    4460 scope.go:122] "RemoveContainer" containerID="ad8fea4da5a956400f0441e12c1dd1bcd7d1c33fb7d78620b3ca63bb0183024f"
Oct 31 21:35:24.577476 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:24.577454    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-7ac3c8-76bc49454f-vfdpw"
Oct 31 21:35:24.591174 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:24.591152    4460 scope.go:122] "RemoveContainer" containerID="ad8fea4da5a956400f0441e12c1dd1bcd7d1c33fb7d78620b3ca63bb0183024f"
Oct 31 21:35:24.591608 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:35:24.591502    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad8fea4da5a956400f0441e12c1dd1bcd7d1c33fb7d78620b3ca63bb0183024f\": container with ID starting with ad8fea4da5a956400f0441e12c1dd1bcd7d1c33fb7d78620b3ca63bb0183024f not found: ID does not exist" containerID="ad8fea4da5a956400f0441e12c1dd1bcd7d1c33fb7d78620b3ca63bb0183024f"
Oct 31 21:35:24.591608 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:24.591551    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad8fea4da5a956400f0441e12c1dd1bcd7d1c33fb7d78620b3ca63bb0183024f"} err="failed to get container status \"ad8fea4da5a956400f0441e12c1dd1bcd7d1c33fb7d78620b3ca63bb0183024f\": rpc error: code = NotFound desc = could not find container \"ad8fea4da5a956400f0441e12c1dd1bcd7d1c33fb7d78620b3ca63bb0183024f\": container with ID starting with ad8fea4da5a956400f0441e12c1dd1bcd7d1c33fb7d78620b3ca63bb0183024f not found: ID does not exist"
Oct 31 21:35:24.639169 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:24.639137    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7ac3c8-76bc49454f-vfdpw"]
Oct 31 21:35:24.646411 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:24.646384    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-7ac3c8-76bc49454f-vfdpw"]
Oct 31 21:35:25.318926 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:25.318888    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-697b9f5476-zz4qf"]
Oct 31 21:35:25.319427 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:25.319163    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-697b9f5476-zz4qf" podUID="9f6b4afa-aaec-48fd-be70-7e69f1d74fc1" containerName="maas-api" containerID="cri-o://a8119e65258091e928f07ef43bffd347dd8690b4e22b343f63b978a6a077fe69" gracePeriod=30
Oct 31 21:35:25.382625 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:25.382582    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:25.618924 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:25.618533    4460 generic.go:356] "Generic (PLEG): container finished" podID="9f6b4afa-aaec-48fd-be70-7e69f1d74fc1" containerID="a8119e65258091e928f07ef43bffd347dd8690b4e22b343f63b978a6a077fe69" exitCode=0
Oct 31 21:35:25.618924 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:25.618804    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-697b9f5476-zz4qf" event={"ID":"9f6b4afa-aaec-48fd-be70-7e69f1d74fc1","Type":"ContainerDied","Data":"a8119e65258091e928f07ef43bffd347dd8690b4e22b343f63b978a6a077fe69"}
Oct 31 21:35:25.855335 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:25.855297    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-697b9f5476-zz4qf"
Oct 31 21:35:25.939036 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:25.928781    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-maas-api-tls\") pod \"9f6b4afa-aaec-48fd-be70-7e69f1d74fc1\" (UID: \"9f6b4afa-aaec-48fd-be70-7e69f1d74fc1\") "
Oct 31 21:35:25.939036 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:25.928909    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-metrics-certs\") pod \"9f6b4afa-aaec-48fd-be70-7e69f1d74fc1\" (UID: \"9f6b4afa-aaec-48fd-be70-7e69f1d74fc1\") "
Oct 31 21:35:25.939036 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:25.928970    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-openshift-service-ca\") pod \"9f6b4afa-aaec-48fd-be70-7e69f1d74fc1\" (UID: \"9f6b4afa-aaec-48fd-be70-7e69f1d74fc1\") "
Oct 31 21:35:25.939036 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:25.929096    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-kube-api-access-sqmgt\" (UniqueName: \"kubernetes.io/projected/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-kube-api-access-sqmgt\") pod \"9f6b4afa-aaec-48fd-be70-7e69f1d74fc1\" (UID: \"9f6b4afa-aaec-48fd-be70-7e69f1d74fc1\") "
Oct 31 21:35:25.955185 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:25.954738    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-kube-api-access-sqmgt" pod "9f6b4afa-aaec-48fd-be70-7e69f1d74fc1" (UID: "9f6b4afa-aaec-48fd-be70-7e69f1d74fc1"). InnerVolumeSpecName "kube-api-access-sqmgt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:35:25.955185 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:25.955111    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-openshift-service-ca" pod "9f6b4afa-aaec-48fd-be70-7e69f1d74fc1" (UID: "9f6b4afa-aaec-48fd-be70-7e69f1d74fc1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:35:26.040966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:26.040538    4460 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-openshift-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:35:26.040966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:26.040587    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqmgt\" (UniqueName: \"kubernetes.io/projected/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-kube-api-access-sqmgt\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:35:26.063965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:26.041784    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-maas-api-tls" pod "9f6b4afa-aaec-48fd-be70-7e69f1d74fc1" (UID: "9f6b4afa-aaec-48fd-be70-7e69f1d74fc1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 21:35:26.093965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:26.086703    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:26.103842 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:26.103796    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-metrics-certs" pod "9f6b4afa-aaec-48fd-be70-7e69f1d74fc1" (UID: "9f6b4afa-aaec-48fd-be70-7e69f1d74fc1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 21:35:26.142834 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:26.142791    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-maas-api-tls\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:35:26.142834 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:26.142831    4460 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1-metrics-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:35:26.198840 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:26.198743    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:26.227553 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:26.227518    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8497c5c1-966e-48c3-beef-d982c8ab7627" path="/var/lib/kubelet/pods/8497c5c1-966e-48c3-beef-d982c8ab7627/volumes"
Oct 31 21:35:26.321983 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:26.316553    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:26.627662 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:26.627629    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-697b9f5476-zz4qf"
Oct 31 21:35:26.627848 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:26.627634    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-697b9f5476-zz4qf" event={"ID":"9f6b4afa-aaec-48fd-be70-7e69f1d74fc1","Type":"ContainerDied","Data":"019726a245d618d0ae978596efeb02a524a17aa81241843a5d65a13f172e5bca"}
Oct 31 21:35:26.627848 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:26.627773    4460 scope.go:122] "RemoveContainer" containerID="a8119e65258091e928f07ef43bffd347dd8690b4e22b343f63b978a6a077fe69"
Oct 31 21:35:26.663157 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:26.663116    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:26.686285 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:26.686248    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-697b9f5476-zz4qf"]
Oct 31 21:35:26.699649 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:26.699613    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-90db52-697b9f5476-zz4qf"]
Oct 31 21:35:28.129744 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:28.129711    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:28.259860 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:28.256718    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9f6b4afa-aaec-48fd-be70-7e69f1d74fc1" path="/var/lib/kubelet/pods/9f6b4afa-aaec-48fd-be70-7e69f1d74fc1/volumes"
Oct 31 21:35:30.012081 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:30.012041    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:30.020333 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:30.020301    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-084d807f-7648554b69-6k7t8"]
Oct 31 21:35:30.054860 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:30.054831    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-084d807f-5ff89dd6bc-6hnlc"]
Oct 31 21:35:30.162066 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:30.162025    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:30.292517 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:30.292424    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:31.720368 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:31.720335    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:33.428769 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:33.428728    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:33.628703 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:33.628668    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:33.795988 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:33.793069    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:34.006912 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:34.006876    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:35.021553 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:35.021512    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-084d807f-7648554b69-6k7t8" podUID="069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3" containerName="payload-processing" containerID="cri-o://39f7762dc53cbf5e562fff9e6a49aecaef1c6612a041064278613d962edebc67" gracePeriod=25
Oct 31 21:35:35.055515 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:35.055482    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-084d807f-5ff89dd6bc-6hnlc" podUID="3c2ada97-edd3-47ec-9d4a-d0c48674685a" containerName="payload-pre-processing" containerID="cri-o://f8b8866a9578a95563b791f01222eac672f7eb6a9cb1d585a95915d91ad32496" gracePeriod=25
Oct 31 21:35:35.170269 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:35.170231    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-084d807f-5ff89dd6bc-6hnlc" podUID="3c2ada97-edd3-47ec-9d4a-d0c48674685a" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 21:35:35.170269 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:35.170269    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-084d807f-7648554b69-6k7t8" podUID="069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 21:35:35.415143 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:35.415059    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:37.167806 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:37.167773    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:38.760723 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:38.760684    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:38.924685 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:38.924633    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:39.421891 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:39.421854    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:39.844274 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:39.844235    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"]
Oct 31 21:35:39.845371 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:39.845341    4460 cpu_manager.go:414] "containerMap: removing container" podUID="9f6b4afa-aaec-48fd-be70-7e69f1d74fc1" containerName="maas-api"
Oct 31 21:35:39.845371 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:39.845370    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9f6b4afa-aaec-48fd-be70-7e69f1d74fc1" containerName="maas-api"
Oct 31 21:35:39.845575 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:39.845447    4460 cpu_manager.go:414] "containerMap: removing container" podUID="8497c5c1-966e-48c3-beef-d982c8ab7627" containerName="payload-processing"
Oct 31 21:35:39.845575 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:39.845455    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8497c5c1-966e-48c3-beef-d982c8ab7627" containerName="payload-processing"
Oct 31 21:35:39.845698 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:39.845628    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="8497c5c1-966e-48c3-beef-d982c8ab7627" containerName="payload-processing"
Oct 31 21:35:39.845698 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:39.845647    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="9f6b4afa-aaec-48fd-be70-7e69f1d74fc1" containerName="maas-api"
Oct 31 21:35:39.851310 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:39.851290    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:35:39.869757 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:39.869729    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"]
Oct 31 21:35:39.879967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:39.879925    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:39.901209 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:39.901182    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-43fb1b-openshift-default-dockercfg-8nsgh\""
Oct 31 21:35:39.972976 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:39.970998    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-workload-socket\") pod \"e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") " pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:35:39.972976 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:39.971080    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd82789d-81ca-487e-8bfb-e2567ca866bd-istiod-ca-cert\") pod \"e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") " pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:35:39.972976 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:39.971136    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd82789d-81ca-487e-8bfb-e2567ca866bd-istio-token\") pod \"e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") " pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:35:39.972976 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:39.971218    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-credential-socket\") pod \"e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") " pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:35:39.972976 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:39.971275    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-istio-data\") pod \"e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") " pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:35:39.972976 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:39.971361    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f976\" (UniqueName: \"kubernetes.io/projected/dd82789d-81ca-487e-8bfb-e2567ca866bd-kube-api-access-7f976\") pod \"e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") " pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:35:39.972976 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:39.971408    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd82789d-81ca-487e-8bfb-e2567ca866bd-istio-podinfo\") pod \"e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") " pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:35:39.972976 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:39.971467    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-workload-certs\") pod \"e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") " pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:35:39.972976 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:39.971594    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-istio-envoy\") pod \"e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") " pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:35:40.078965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:40.076247    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd82789d-81ca-487e-8bfb-e2567ca866bd-istiod-ca-cert\") pod \"e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") " pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:35:40.078965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:40.077296    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd82789d-81ca-487e-8bfb-e2567ca866bd-istiod-ca-cert\") pod \"e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") " pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:35:40.078965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:40.077395    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd82789d-81ca-487e-8bfb-e2567ca866bd-istio-token\") pod \"e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") " pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:35:40.078965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:40.077756    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-credential-socket\") pod \"e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") " pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:35:40.078965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:40.077804    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-istio-data\") pod \"e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") " pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:35:40.078965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:40.077879    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f976\" (UniqueName: \"kubernetes.io/projected/dd82789d-81ca-487e-8bfb-e2567ca866bd-kube-api-access-7f976\") pod \"e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") " pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:35:40.079421 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:40.077924    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd82789d-81ca-487e-8bfb-e2567ca866bd-istio-podinfo\") pod \"e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") " pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:35:40.079421 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:40.078225    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-istio-data\") pod \"e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") " pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:35:40.079421 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:40.079087    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-credential-socket\") pod \"e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") " pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:35:40.079421 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:40.079340    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-workload-certs\") pod \"e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") " pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:35:40.079965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:40.079713    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-istio-envoy\") pod \"e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") " pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:35:40.080438 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:40.080385    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-workload-socket\") pod \"e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") " pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:35:40.080740 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:40.080414    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-workload-certs\") pod \"e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") " pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:35:40.080929 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:40.080860    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-workload-socket\") pod \"e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") " pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:35:40.113016 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:40.109484    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-istio-envoy\") pod \"e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") " pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:35:40.121958 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:40.120070    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd82789d-81ca-487e-8bfb-e2567ca866bd-istio-token\") pod \"e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") " pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:35:40.134002 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:40.133707    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd82789d-81ca-487e-8bfb-e2567ca866bd-istio-podinfo\") pod \"e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") " pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:35:40.137789 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:40.137722    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f976\" (UniqueName: \"kubernetes.io/projected/dd82789d-81ca-487e-8bfb-e2567ca866bd-kube-api-access-7f976\") pod \"e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") " pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:35:40.172320 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:40.172281    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:35:40.277671 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:40.277433    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:40.337669 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:40.337634    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:35:40.643020 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:40.640283    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"]
Oct 31 21:35:40.643020 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:35:40.640444    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd82789d_81ca_487e_8bfb_e2567ca866bd.slice/crio-94f630e217c04b28264a43bb6368ae49af3c4331d81c417b8fb979abb75d0a6f WatchSource:0}: Error finding container 94f630e217c04b28264a43bb6368ae49af3c4331d81c417b8fb979abb75d0a6f: Status 404 returned error can't find the container with id 94f630e217c04b28264a43bb6368ae49af3c4331d81c417b8fb979abb75d0a6f
Oct 31 21:35:40.681053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:40.681007    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:35:40.681266 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:40.681103    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:35:40.681266 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:40.681149    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:35:40.739455 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:40.739419    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs" event={"ID":"dd82789d-81ca-487e-8bfb-e2567ca866bd","Type":"ContainerStarted","Data":"94f630e217c04b28264a43bb6368ae49af3c4331d81c417b8fb979abb75d0a6f"}
Oct 31 21:35:40.772126 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:40.772097    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:41.748400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:41.748363    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs" event={"ID":"dd82789d-81ca-487e-8bfb-e2567ca866bd","Type":"ContainerStarted","Data":"dc44e026b70ed551abcbbbc17389ec363ff129c85ceeb95bb992464be87eabdb"}
Oct 31 21:35:41.811146 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:41.811087    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs" podStartSLOduration=2.811068734 podStartE2EDuration="2.811068734s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:35:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:35:41.809995664 +0000 UTC m=+1181.726305095" watchObservedRunningTime="2026-10-31 21:35:41.811068734 +0000 UTC m=+1181.727378165"
Oct 31 21:35:42.174975 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:42.174918    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:35:42.181437 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:42.181409    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:35:42.645632 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:42.645593    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:42.784223 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:42.784192    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:35:42.787016 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:42.786975    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:35:44.150768 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:44.150084    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:44.257422 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:44.257391    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:44.497539 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:44.497440    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:45.149878 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:45.149837    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-371531-7b5cd97689-rtfrd"]
Oct 31 21:35:45.170786 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:45.170753    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-084d807f-7648554b69-6k7t8" podUID="069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 21:35:45.171166 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:45.170764    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-084d807f-5ff89dd6bc-6hnlc" podUID="3c2ada97-edd3-47ec-9d4a-d0c48674685a" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 21:35:45.314569 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:45.314524    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:45.438315 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:45.437495    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:45.865781 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:45.865752    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:45.980005 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:45.979970    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:46.740522 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:46.740466    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:46.897798 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:46.897759    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:47.147557 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:47.147518    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-43fb1b-5799d848d7-m7b45"]
Oct 31 21:35:47.152399 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:47.152368    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-43fb1b-5799d848d7-m7b45"
Oct 31 21:35:47.158062 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:47.158028    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-43fb1b-5799d848d7-m7b45"]
Oct 31 21:35:47.168913 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:47.168894    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-43fb1b\""
Oct 31 21:35:47.194964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:47.192566    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-43fb1b-dockercfg-nf24q\""
Oct 31 21:35:47.358962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:47.357906    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzw85\" (UniqueName: \"kubernetes.io/projected/850eb5d3-4057-4faa-a3ed-4b7e95f31108-kube-api-access-nzw85\") pod \"payload-pre-processing-e2e-api-b-43fb1b-5799d848d7-m7b45\" (UID: \"850eb5d3-4057-4faa-a3ed-4b7e95f31108\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-43fb1b-5799d848d7-m7b45"
Oct 31 21:35:47.358962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:47.358046    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/850eb5d3-4057-4faa-a3ed-4b7e95f31108-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-43fb1b-5799d848d7-m7b45\" (UID: \"850eb5d3-4057-4faa-a3ed-4b7e95f31108\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-43fb1b-5799d848d7-m7b45"
Oct 31 21:35:47.466590 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:47.466119    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzw85\" (UniqueName: \"kubernetes.io/projected/850eb5d3-4057-4faa-a3ed-4b7e95f31108-kube-api-access-nzw85\") pod \"payload-pre-processing-e2e-api-b-43fb1b-5799d848d7-m7b45\" (UID: \"850eb5d3-4057-4faa-a3ed-4b7e95f31108\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-43fb1b-5799d848d7-m7b45"
Oct 31 21:35:47.491386 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:47.489441    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/850eb5d3-4057-4faa-a3ed-4b7e95f31108-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-43fb1b-5799d848d7-m7b45\" (UID: \"850eb5d3-4057-4faa-a3ed-4b7e95f31108\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-43fb1b-5799d848d7-m7b45"
Oct 31 21:35:47.531974 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:47.495799    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzw85\" (UniqueName: \"kubernetes.io/projected/850eb5d3-4057-4faa-a3ed-4b7e95f31108-kube-api-access-nzw85\") pod \"payload-pre-processing-e2e-api-b-43fb1b-5799d848d7-m7b45\" (UID: \"850eb5d3-4057-4faa-a3ed-4b7e95f31108\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-43fb1b-5799d848d7-m7b45"
Oct 31 21:35:47.535804 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:47.535773    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/850eb5d3-4057-4faa-a3ed-4b7e95f31108-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-43fb1b-5799d848d7-m7b45\" (UID: \"850eb5d3-4057-4faa-a3ed-4b7e95f31108\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-43fb1b-5799d848d7-m7b45"
Oct 31 21:35:47.770740 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:47.770707    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-43fb1b-5799d848d7-m7b45"
Oct 31 21:35:47.911135 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:47.911097    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:48.099587 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:48.099561    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-43fb1b-5799d848d7-m7b45"]
Oct 31 21:35:48.102402 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:35:48.102376    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod850eb5d3_4057_4faa_a3ed_4b7e95f31108.slice/crio-828dab523690524d0597f32df22c04f3704677999fbc16dec0a448c4c7b52158 WatchSource:0}: Error finding container 828dab523690524d0597f32df22c04f3704677999fbc16dec0a448c4c7b52158: Status 404 returned error can't find the container with id 828dab523690524d0597f32df22c04f3704677999fbc16dec0a448c4c7b52158
Oct 31 21:35:48.912807 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:48.912761    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-43fb1b-5799d848d7-m7b45" event={"ID":"850eb5d3-4057-4faa-a3ed-4b7e95f31108","Type":"ContainerStarted","Data":"05ef485fd64dc230102d0cd36ad4acf2d5412b8969b3c88e829a049fb39da3ca"}
Oct 31 21:35:48.913330 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:48.912830    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-43fb1b-5799d848d7-m7b45" event={"ID":"850eb5d3-4057-4faa-a3ed-4b7e95f31108","Type":"ContainerStarted","Data":"828dab523690524d0597f32df22c04f3704677999fbc16dec0a448c4c7b52158"}
Oct 31 21:35:48.913330 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:48.912856    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-43fb1b-5799d848d7-m7b45"
Oct 31 21:35:48.947651 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:48.947621    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:48.958758 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:48.958707    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-43fb1b-5799d848d7-m7b45" podStartSLOduration=1.958693606 podStartE2EDuration="1.958693606s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:35:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:35:48.956144841 +0000 UTC m=+1188.872454271" watchObservedRunningTime="2026-10-31 21:35:48.958693606 +0000 UTC m=+1188.875003036"
Oct 31 21:35:50.150984 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:50.150928    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-371531-7b5cd97689-rtfrd" podUID="9bcfc171-925d-4bfd-a2b6-61c8b90bc395" containerName="payload-processing" containerID="cri-o://3ed043e251d1c0964a5992ec138584f540def3f560856f241c7788053540e614" gracePeriod=25
Oct 31 21:35:50.447152 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:50.447124    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-371531-7b5cd97689-rtfrd"
Oct 31 21:35:50.556822 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:50.556783    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9bcfc171-925d-4bfd-a2b6-61c8b90bc395-kube-api-access-6vdwt\" (UniqueName: \"kubernetes.io/projected/9bcfc171-925d-4bfd-a2b6-61c8b90bc395-kube-api-access-6vdwt\") pod \"9bcfc171-925d-4bfd-a2b6-61c8b90bc395\" (UID: \"9bcfc171-925d-4bfd-a2b6-61c8b90bc395\") "
Oct 31 21:35:50.557177 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:50.557155    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9bcfc171-925d-4bfd-a2b6-61c8b90bc395-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bcfc171-925d-4bfd-a2b6-61c8b90bc395-plugins-config-volume\") pod \"9bcfc171-925d-4bfd-a2b6-61c8b90bc395\" (UID: \"9bcfc171-925d-4bfd-a2b6-61c8b90bc395\") "
Oct 31 21:35:50.557513 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:50.557489    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bcfc171-925d-4bfd-a2b6-61c8b90bc395-plugins-config-volume" pod "9bcfc171-925d-4bfd-a2b6-61c8b90bc395" (UID: "9bcfc171-925d-4bfd-a2b6-61c8b90bc395"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:35:50.557769 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:50.557753    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9bcfc171-925d-4bfd-a2b6-61c8b90bc395-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:35:50.558987 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:50.558958    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bcfc171-925d-4bfd-a2b6-61c8b90bc395-kube-api-access-6vdwt" pod "9bcfc171-925d-4bfd-a2b6-61c8b90bc395" (UID: "9bcfc171-925d-4bfd-a2b6-61c8b90bc395"). InnerVolumeSpecName "kube-api-access-6vdwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:35:50.659499 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:50.659469    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vdwt\" (UniqueName: \"kubernetes.io/projected/9bcfc171-925d-4bfd-a2b6-61c8b90bc395-kube-api-access-6vdwt\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:35:50.724832 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:50.724732    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:50.933164 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:50.933132    4460 generic.go:356] "Generic (PLEG): container finished" podID="9bcfc171-925d-4bfd-a2b6-61c8b90bc395" containerID="3ed043e251d1c0964a5992ec138584f540def3f560856f241c7788053540e614" exitCode=0
Oct 31 21:35:50.933312 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:50.933179    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-371531-7b5cd97689-rtfrd" event={"ID":"9bcfc171-925d-4bfd-a2b6-61c8b90bc395","Type":"ContainerDied","Data":"3ed043e251d1c0964a5992ec138584f540def3f560856f241c7788053540e614"}
Oct 31 21:35:50.933312 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:50.933184    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-371531-7b5cd97689-rtfrd"
Oct 31 21:35:50.933312 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:50.933224    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-371531-7b5cd97689-rtfrd" event={"ID":"9bcfc171-925d-4bfd-a2b6-61c8b90bc395","Type":"ContainerDied","Data":"4a85cb22328647816280050b0440ad0698605d2dd30413a000e8abe627c6196f"}
Oct 31 21:35:50.933312 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:50.933250    4460 scope.go:122] "RemoveContainer" containerID="3ed043e251d1c0964a5992ec138584f540def3f560856f241c7788053540e614"
Oct 31 21:35:50.948538 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:50.948517    4460 scope.go:122] "RemoveContainer" containerID="3ed043e251d1c0964a5992ec138584f540def3f560856f241c7788053540e614"
Oct 31 21:35:50.948804 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:35:50.948785    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ed043e251d1c0964a5992ec138584f540def3f560856f241c7788053540e614\": container with ID starting with 3ed043e251d1c0964a5992ec138584f540def3f560856f241c7788053540e614 not found: ID does not exist" containerID="3ed043e251d1c0964a5992ec138584f540def3f560856f241c7788053540e614"
Oct 31 21:35:50.948886 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:50.948812    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ed043e251d1c0964a5992ec138584f540def3f560856f241c7788053540e614"} err="failed to get container status \"3ed043e251d1c0964a5992ec138584f540def3f560856f241c7788053540e614\": rpc error: code = NotFound desc = could not find container \"3ed043e251d1c0964a5992ec138584f540def3f560856f241c7788053540e614\": container with ID starting with 3ed043e251d1c0964a5992ec138584f540def3f560856f241c7788053540e614 not found: ID does not exist"
Oct 31 21:35:50.992248 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:50.992172    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-371531-7b5cd97689-rtfrd"]
Oct 31 21:35:51.004786 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:51.004758    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-371531-7b5cd97689-rtfrd"]
Oct 31 21:35:51.455550 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:35:51.455518    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:35:52.227808 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:52.227769    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9bcfc171-925d-4bfd-a2b6-61c8b90bc395" path="/var/lib/kubelet/pods/9bcfc171-925d-4bfd-a2b6-61c8b90bc395/volumes"
Oct 31 21:35:52.448167 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:52.448077    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:52.950594 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:52.950561    4460 generic.go:356] "Generic (PLEG): container finished" podID="485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5" containerID="10cbadfd2d786bbb06634b3e2368ad837f3385778fe526b74cc998e38dcfc6c4" exitCode=0
Oct 31 21:35:52.951027 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:52.950633    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z" event={"ID":"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5","Type":"ContainerDied","Data":"10cbadfd2d786bbb06634b3e2368ad837f3385778fe526b74cc998e38dcfc6c4"}
Oct 31 21:35:52.951116 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:52.951086    4460 scope.go:122] "RemoveContainer" containerID="10cbadfd2d786bbb06634b3e2368ad837f3385778fe526b74cc998e38dcfc6c4"
Oct 31 21:35:52.952545 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:52.952519    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:35:52.952627 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:52.952584    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:35:52.952627 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:52.952617    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:35:53.959660 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:53.959615    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z" event={"ID":"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5","Type":"ContainerStarted","Data":"9a712bedde325120c9746e740297a44c2df1db8e77e85dc6e2a463e8736a2782"}
Oct 31 21:35:54.090887 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:54.090852    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:35:54.091826 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:54.091798    4460 patch_prober.go:29] interesting pod/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 31 21:35:54.091975 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:54.091853    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z" podUID="485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 31 21:35:54.156790 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:54.156755    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:54.598760 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:54.598717    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:54.729986 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:54.724556    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:54.994420 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:54.994084    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:55.090881 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:55.090850    4460 patch_prober.go:29] interesting pod/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 31 21:35:55.091046 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:55.090902    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z" podUID="485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 31 21:35:55.203966 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:35:55.194058    4460 logging.go:55] [core] [Channel #1433 SubChannel #1434]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.83:50052", ServerName: "10.133.0.83:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.83:50052: connect: connection refused"
Oct 31 21:35:55.232976 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:35:55.213427    4460 logging.go:55] [core] [Channel #1435 SubChannel #1436]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.84:50052", ServerName: "10.133.0.84:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.84:50052: connect: connection refused"
Oct 31 21:35:55.613215 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:55.611157    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-084d807f-7648554b69-6k7t8"
Oct 31 21:35:55.705271 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:55.705244    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-084d807f-5ff89dd6bc-6hnlc"
Oct 31 21:35:55.747869 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:55.747842    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3-kube-api-access-gztpf\" (UniqueName: \"kubernetes.io/projected/069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3-kube-api-access-gztpf\") pod \"069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3\" (UID: \"069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3\") "
Oct 31 21:35:55.748016 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:55.747976    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3-plugins-config-volume\") pod \"069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3\" (UID: \"069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3\") "
Oct 31 21:35:55.748340 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:55.748316    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3-plugins-config-volume" pod "069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3" (UID: "069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:35:55.748626 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:55.748606    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:35:55.750209 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:55.750187    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3-kube-api-access-gztpf" pod "069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3" (UID: "069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3"). InnerVolumeSpecName "kube-api-access-gztpf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:35:55.849960 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:55.849856    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c2ada97-edd3-47ec-9d4a-d0c48674685a-kube-api-access-f7gb2\" (UniqueName: \"kubernetes.io/projected/3c2ada97-edd3-47ec-9d4a-d0c48674685a-kube-api-access-f7gb2\") pod \"3c2ada97-edd3-47ec-9d4a-d0c48674685a\" (UID: \"3c2ada97-edd3-47ec-9d4a-d0c48674685a\") "
Oct 31 21:35:55.850345 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:55.850312    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3c2ada97-edd3-47ec-9d4a-d0c48674685a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c2ada97-edd3-47ec-9d4a-d0c48674685a-plugins-config-volume\") pod \"3c2ada97-edd3-47ec-9d4a-d0c48674685a\" (UID: \"3c2ada97-edd3-47ec-9d4a-d0c48674685a\") "
Oct 31 21:35:55.850689 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:55.850664    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c2ada97-edd3-47ec-9d4a-d0c48674685a-plugins-config-volume" pod "3c2ada97-edd3-47ec-9d4a-d0c48674685a" (UID: "3c2ada97-edd3-47ec-9d4a-d0c48674685a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:35:55.850987 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:55.850932    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c2ada97-edd3-47ec-9d4a-d0c48674685a-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:35:55.851074 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:55.850991    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gztpf\" (UniqueName: \"kubernetes.io/projected/069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3-kube-api-access-gztpf\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:35:55.852147 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:55.852124    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c2ada97-edd3-47ec-9d4a-d0c48674685a-kube-api-access-f7gb2" pod "3c2ada97-edd3-47ec-9d4a-d0c48674685a" (UID: "3c2ada97-edd3-47ec-9d4a-d0c48674685a"). InnerVolumeSpecName "kube-api-access-f7gb2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:35:55.953166 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:55.953130    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7gb2\" (UniqueName: \"kubernetes.io/projected/3c2ada97-edd3-47ec-9d4a-d0c48674685a-kube-api-access-f7gb2\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:35:56.016319 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:56.016282    4460 generic.go:356] "Generic (PLEG): container finished" podID="3c2ada97-edd3-47ec-9d4a-d0c48674685a" containerID="f8b8866a9578a95563b791f01222eac672f7eb6a9cb1d585a95915d91ad32496" exitCode=0
Oct 31 21:35:56.016757 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:56.016350    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-084d807f-5ff89dd6bc-6hnlc"
Oct 31 21:35:56.016757 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:56.016363    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-084d807f-5ff89dd6bc-6hnlc" event={"ID":"3c2ada97-edd3-47ec-9d4a-d0c48674685a","Type":"ContainerDied","Data":"f8b8866a9578a95563b791f01222eac672f7eb6a9cb1d585a95915d91ad32496"}
Oct 31 21:35:56.016757 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:56.016442    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-084d807f-5ff89dd6bc-6hnlc" event={"ID":"3c2ada97-edd3-47ec-9d4a-d0c48674685a","Type":"ContainerDied","Data":"f8ca8c8524e09c8a9c6c51b3ca06a1a979796aa2df0131fa2af239c96c1bc55e"}
Oct 31 21:35:56.016757 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:56.016465    4460 scope.go:122] "RemoveContainer" containerID="f8b8866a9578a95563b791f01222eac672f7eb6a9cb1d585a95915d91ad32496"
Oct 31 21:35:56.018126 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:56.018096    4460 generic.go:356] "Generic (PLEG): container finished" podID="069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3" containerID="39f7762dc53cbf5e562fff9e6a49aecaef1c6612a041064278613d962edebc67" exitCode=0
Oct 31 21:35:56.018236 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:56.018172    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-084d807f-7648554b69-6k7t8"
Oct 31 21:35:56.018301 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:56.018279    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-084d807f-7648554b69-6k7t8" event={"ID":"069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3","Type":"ContainerDied","Data":"39f7762dc53cbf5e562fff9e6a49aecaef1c6612a041064278613d962edebc67"}
Oct 31 21:35:56.018360 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:56.018326    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-084d807f-7648554b69-6k7t8" event={"ID":"069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3","Type":"ContainerDied","Data":"c45555d8d893a606e05fbd7d4fcc71c299c28d7570c270883f565b4dae97f7c5"}
Oct 31 21:35:56.053168 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:56.053144    4460 scope.go:122] "RemoveContainer" containerID="f8b8866a9578a95563b791f01222eac672f7eb6a9cb1d585a95915d91ad32496"
Oct 31 21:35:56.053579 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:35:56.053491    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8b8866a9578a95563b791f01222eac672f7eb6a9cb1d585a95915d91ad32496\": container with ID starting with f8b8866a9578a95563b791f01222eac672f7eb6a9cb1d585a95915d91ad32496 not found: ID does not exist" containerID="f8b8866a9578a95563b791f01222eac672f7eb6a9cb1d585a95915d91ad32496"
Oct 31 21:35:56.053579 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:56.053531    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8b8866a9578a95563b791f01222eac672f7eb6a9cb1d585a95915d91ad32496"} err="failed to get container status \"f8b8866a9578a95563b791f01222eac672f7eb6a9cb1d585a95915d91ad32496\": rpc error: code = NotFound desc = could not find container \"f8b8866a9578a95563b791f01222eac672f7eb6a9cb1d585a95915d91ad32496\": container with ID starting with f8b8866a9578a95563b791f01222eac672f7eb6a9cb1d585a95915d91ad32496 not found: ID does not exist"
Oct 31 21:35:56.053579 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:56.053549    4460 scope.go:122] "RemoveContainer" containerID="39f7762dc53cbf5e562fff9e6a49aecaef1c6612a041064278613d962edebc67"
Oct 31 21:35:56.068762 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:56.068741    4460 scope.go:122] "RemoveContainer" containerID="39f7762dc53cbf5e562fff9e6a49aecaef1c6612a041064278613d962edebc67"
Oct 31 21:35:56.069071 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:35:56.069051    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39f7762dc53cbf5e562fff9e6a49aecaef1c6612a041064278613d962edebc67\": container with ID starting with 39f7762dc53cbf5e562fff9e6a49aecaef1c6612a041064278613d962edebc67 not found: ID does not exist" containerID="39f7762dc53cbf5e562fff9e6a49aecaef1c6612a041064278613d962edebc67"
Oct 31 21:35:56.069130 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:56.069078    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39f7762dc53cbf5e562fff9e6a49aecaef1c6612a041064278613d962edebc67"} err="failed to get container status \"39f7762dc53cbf5e562fff9e6a49aecaef1c6612a041064278613d962edebc67\": rpc error: code = NotFound desc = could not find container \"39f7762dc53cbf5e562fff9e6a49aecaef1c6612a041064278613d962edebc67\": container with ID starting with 39f7762dc53cbf5e562fff9e6a49aecaef1c6612a041064278613d962edebc67 not found: ID does not exist"
Oct 31 21:35:56.091431 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:56.091408    4460 patch_prober.go:29] interesting pod/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 31 21:35:56.091593 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:56.091455    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z" podUID="485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 31 21:35:56.108492 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:56.108435    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-084d807f-5ff89dd6bc-6hnlc"]
Oct 31 21:35:56.114759 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:56.114738    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-084d807f-5ff89dd6bc-6hnlc"]
Oct 31 21:35:56.148509 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:56.148473    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:56.202815 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:56.201520    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-084d807f-5ff89dd6bc-6hnlc" podUID="3c2ada97-edd3-47ec-9d4a-d0c48674685a" containerName="payload-pre-processing" probeResult="failure" output="timeout: failed to connect service \"10.133.0.83:50052\" within 1s: context deadline exceeded"
Oct 31 21:35:56.202815 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:35:56.201773    4460 logging.go:55] [core] [Channel #1433 SubChannel #1434]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.83:50052", ServerName: "10.133.0.83:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.83:50052: operation was canceled"
Oct 31 21:35:56.218996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:56.216002    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-084d807f-7648554b69-6k7t8"]
Oct 31 21:35:56.218996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:56.216306    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-084d807f-7648554b69-6k7t8" podUID="069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.133.0.84:50052\" within 1s: context deadline exceeded"
Oct 31 21:35:56.218996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:56.216400    4460 status_manager.go:453] "Container readiness changed before pod has synced" pod="openshift-ingress/payload-processing-e2e-mt-084d807f-7648554b69-6k7t8" containerID="cri-o://39f7762dc53cbf5e562fff9e6a49aecaef1c6612a041064278613d962edebc67"
Oct 31 21:35:56.218996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:56.216418    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-084d807f-7648554b69-6k7t8"
Oct 31 21:35:56.278980 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:56.269749    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3c2ada97-edd3-47ec-9d4a-d0c48674685a" path="/var/lib/kubelet/pods/3c2ada97-edd3-47ec-9d4a-d0c48674685a/volumes"
Oct 31 21:35:56.278980 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:56.272632    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-084d807f-7648554b69-6k7t8"]
Oct 31 21:35:56.808213 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:56.808174    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:35:56.814949 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:56.814916    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:57.091482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:57.091403    4460 patch_prober.go:29] interesting pod/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 31 21:35:57.091482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:57.091459    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z" podUID="485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 31 21:35:57.108009 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:57.107977    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:58.091550 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:58.091517    4460 patch_prober.go:29] interesting pod/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 31 21:35:58.091969 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:58.091570    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z" podUID="485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 31 21:35:58.112118 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:58.112091    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:58.258461 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:58.258421    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3" path="/var/lib/kubelet/pods/069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3/volumes"
Oct 31 21:35:59.091320 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:59.091284    4460 patch_prober.go:29] interesting pod/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused" start-of-body=
Oct 31 21:35:59.091503 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:59.091342    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z" podUID="485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.93:15021/healthz/ready\": dial tcp 10.133.0.93:15021: connect: connection refused"
Oct 31 21:35:59.800407 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:59.800374    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-43fb1b-946d9c597-ddppm"]
Oct 31 21:35:59.801067 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:59.801051    4460 cpu_manager.go:414] "containerMap: removing container" podUID="9bcfc171-925d-4bfd-a2b6-61c8b90bc395" containerName="payload-processing"
Oct 31 21:35:59.801067 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:59.801068    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9bcfc171-925d-4bfd-a2b6-61c8b90bc395" containerName="payload-processing"
Oct 31 21:35:59.801156 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:59.801093    4460 cpu_manager.go:414] "containerMap: removing container" podUID="3c2ada97-edd3-47ec-9d4a-d0c48674685a" containerName="payload-pre-processing"
Oct 31 21:35:59.801156 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:59.801099    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3c2ada97-edd3-47ec-9d4a-d0c48674685a" containerName="payload-pre-processing"
Oct 31 21:35:59.801554 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:59.801271    4460 cpu_manager.go:414] "containerMap: removing container" podUID="069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3" containerName="payload-processing"
Oct 31 21:35:59.801646 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:59.801556    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3" containerName="payload-processing"
Oct 31 21:35:59.802178 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:59.802158    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="3c2ada97-edd3-47ec-9d4a-d0c48674685a" containerName="payload-pre-processing"
Oct 31 21:35:59.802263 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:59.802184    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="069adf89-8ed5-4b2b-b5d1-ea0df0eea3e3" containerName="payload-processing"
Oct 31 21:35:59.802263 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:59.802215    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="9bcfc171-925d-4bfd-a2b6-61c8b90bc395" containerName="payload-processing"
Oct 31 21:35:59.811737 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:59.811707    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-43fb1b-946d9c597-ddppm"
Oct 31 21:35:59.832213 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:59.832180    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-43fb1b-946d9c597-ddppm"]
Oct 31 21:35:59.838999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:59.838976    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-43fb1b-serving-cert\""
Oct 31 21:35:59.885584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:59.885549    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:35:59.924723 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:59.924696    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-43fb1b-5799d848d7-m7b45"
Oct 31 21:35:59.957529 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:59.957500    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c800a8c6-2786-4aaf-ade9-07a46af9cc94-metrics-certs\") pod \"maas-api-e2e-api-b-43fb1b-946d9c597-ddppm\" (UID: \"c800a8c6-2786-4aaf-ade9-07a46af9cc94\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-43fb1b-946d9c597-ddppm"
Oct 31 21:35:59.957650 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:59.957538    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c800a8c6-2786-4aaf-ade9-07a46af9cc94-openshift-service-ca\") pod \"maas-api-e2e-api-b-43fb1b-946d9c597-ddppm\" (UID: \"c800a8c6-2786-4aaf-ade9-07a46af9cc94\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-43fb1b-946d9c597-ddppm"
Oct 31 21:35:59.957650 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:59.957630    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgw25\" (UniqueName: \"kubernetes.io/projected/c800a8c6-2786-4aaf-ade9-07a46af9cc94-kube-api-access-pgw25\") pod \"maas-api-e2e-api-b-43fb1b-946d9c597-ddppm\" (UID: \"c800a8c6-2786-4aaf-ade9-07a46af9cc94\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-43fb1b-946d9c597-ddppm"
Oct 31 21:35:59.957795 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:59.957783    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c800a8c6-2786-4aaf-ade9-07a46af9cc94-maas-api-tls\") pod \"maas-api-e2e-api-b-43fb1b-946d9c597-ddppm\" (UID: \"c800a8c6-2786-4aaf-ade9-07a46af9cc94\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-43fb1b-946d9c597-ddppm"
Oct 31 21:35:59.997571 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:35:59.997533    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:00.059148 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:00.059077    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgw25\" (UniqueName: \"kubernetes.io/projected/c800a8c6-2786-4aaf-ade9-07a46af9cc94-kube-api-access-pgw25\") pod \"maas-api-e2e-api-b-43fb1b-946d9c597-ddppm\" (UID: \"c800a8c6-2786-4aaf-ade9-07a46af9cc94\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-43fb1b-946d9c597-ddppm"
Oct 31 21:36:00.059279 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:00.059261    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c800a8c6-2786-4aaf-ade9-07a46af9cc94-maas-api-tls\") pod \"maas-api-e2e-api-b-43fb1b-946d9c597-ddppm\" (UID: \"c800a8c6-2786-4aaf-ade9-07a46af9cc94\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-43fb1b-946d9c597-ddppm"
Oct 31 21:36:00.059359 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:00.059346    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c800a8c6-2786-4aaf-ade9-07a46af9cc94-metrics-certs\") pod \"maas-api-e2e-api-b-43fb1b-946d9c597-ddppm\" (UID: \"c800a8c6-2786-4aaf-ade9-07a46af9cc94\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-43fb1b-946d9c597-ddppm"
Oct 31 21:36:00.059393 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:00.059375    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c800a8c6-2786-4aaf-ade9-07a46af9cc94-openshift-service-ca\") pod \"maas-api-e2e-api-b-43fb1b-946d9c597-ddppm\" (UID: \"c800a8c6-2786-4aaf-ade9-07a46af9cc94\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-43fb1b-946d9c597-ddppm"
Oct 31 21:36:00.060142 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:00.060114    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c800a8c6-2786-4aaf-ade9-07a46af9cc94-openshift-service-ca\") pod \"maas-api-e2e-api-b-43fb1b-946d9c597-ddppm\" (UID: \"c800a8c6-2786-4aaf-ade9-07a46af9cc94\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-43fb1b-946d9c597-ddppm"
Oct 31 21:36:00.061722 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:00.061700    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c800a8c6-2786-4aaf-ade9-07a46af9cc94-metrics-certs\") pod \"maas-api-e2e-api-b-43fb1b-946d9c597-ddppm\" (UID: \"c800a8c6-2786-4aaf-ade9-07a46af9cc94\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-43fb1b-946d9c597-ddppm"
Oct 31 21:36:00.061860 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:00.061825    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c800a8c6-2786-4aaf-ade9-07a46af9cc94-maas-api-tls\") pod \"maas-api-e2e-api-b-43fb1b-946d9c597-ddppm\" (UID: \"c800a8c6-2786-4aaf-ade9-07a46af9cc94\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-43fb1b-946d9c597-ddppm"
Oct 31 21:36:00.110965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:00.087902    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgw25\" (UniqueName: \"kubernetes.io/projected/c800a8c6-2786-4aaf-ade9-07a46af9cc94-kube-api-access-pgw25\") pod \"maas-api-e2e-api-b-43fb1b-946d9c597-ddppm\" (UID: \"c800a8c6-2786-4aaf-ade9-07a46af9cc94\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-43fb1b-946d9c597-ddppm"
Oct 31 21:36:00.138971 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:00.137281    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:36:00.147983 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:00.139554    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-43fb1b-946d9c597-ddppm"
Oct 31 21:36:00.154549 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:00.150711    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:36:00.450080 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:00.449992    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ssl9_26069598-5dc4-4c96-b55f-dd2f09884419/ovn-acl-logging/0.log"
Oct 31 21:36:00.452045 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:00.452015    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ssl9_26069598-5dc4-4c96-b55f-dd2f09884419/ovn-acl-logging/0.log"
Oct 31 21:36:00.568614 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:00.568583    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-43fb1b-946d9c597-ddppm"]
Oct 31 21:36:00.569230 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:36:00.569201    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc800a8c6_2786_4aaf_ade9_07a46af9cc94.slice/crio-7e0e95b763a9caa2a12af7b79191978eb891b629fcd9062f0fafa90354a91b8a WatchSource:0}: Error finding container 7e0e95b763a9caa2a12af7b79191978eb891b629fcd9062f0fafa90354a91b8a: Status 404 returned error can't find the container with id 7e0e95b763a9caa2a12af7b79191978eb891b629fcd9062f0fafa90354a91b8a
Oct 31 21:36:01.104786 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:01.104546    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-43fb1b-946d9c597-ddppm" event={"ID":"c800a8c6-2786-4aaf-ade9-07a46af9cc94","Type":"ContainerStarted","Data":"7e0e95b763a9caa2a12af7b79191978eb891b629fcd9062f0fafa90354a91b8a"}
Oct 31 21:36:01.609093 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:01.609058    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:02.112455 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:02.112410    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-43fb1b-946d9c597-ddppm" event={"ID":"c800a8c6-2786-4aaf-ade9-07a46af9cc94","Type":"ContainerStarted","Data":"9574a25ecd4f5cdee33275ed86cf26ac835f3fe1e2bf9bfa75ebcb172fca9e3a"}
Oct 31 21:36:02.112917 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:02.112481    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-43fb1b-946d9c597-ddppm"
Oct 31 21:36:02.159733 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:02.159681    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-43fb1b-946d9c597-ddppm" podStartSLOduration=2.6070677939999998 podStartE2EDuration="3.15966673s" totalImagesPullingTime="552.598936ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:35:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:36:02.158693335 +0000 UTC m=+1202.075002777" watchObservedRunningTime="2026-10-31 21:36:02.15966673 +0000 UTC m=+1202.075976160"
Oct 31 21:36:03.213253 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:03.213212    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:03.618301 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:03.618259    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:03.791139 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:03.790996    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:03.927227 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:03.926757    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:04.371163 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:04.371125    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:04.484646 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:04.484608    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:04.616039 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:04.609452    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:05.037697 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:05.037658    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:05.985350 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:05.985319    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-248b29-7c8fd468f5-fm992"]
Oct 31 21:36:06.166710 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:06.166676    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:06.306640 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:06.306593    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:06.417273 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:06.417235    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:08.013078 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:08.013041    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:08.133969 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:08.133839    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:08.144961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:08.141444    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-43fb1b-946d9c597-ddppm"
Oct 31 21:36:09.727909 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:09.727873    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:10.985911 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:10.985855    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-248b29-7c8fd468f5-fm992" podUID="a9280996-adf2-439c-934a-e8fa3aac2ef2" containerName="payload-processing" containerID="cri-o://7773354f12bb59f9635d17d24402abb89d6306670563ee788362db1ee9ea127c" gracePeriod=25
Oct 31 21:36:11.256785 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:11.256711    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-248b29-7c8fd468f5-fm992"
Oct 31 21:36:11.347956 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:11.347908    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a9280996-adf2-439c-934a-e8fa3aac2ef2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9280996-adf2-439c-934a-e8fa3aac2ef2-plugins-config-volume\") pod \"a9280996-adf2-439c-934a-e8fa3aac2ef2\" (UID: \"a9280996-adf2-439c-934a-e8fa3aac2ef2\") "
Oct 31 21:36:11.348272 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:11.348247    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9280996-adf2-439c-934a-e8fa3aac2ef2-kube-api-access-8x7wm\" (UniqueName: \"kubernetes.io/projected/a9280996-adf2-439c-934a-e8fa3aac2ef2-kube-api-access-8x7wm\") pod \"a9280996-adf2-439c-934a-e8fa3aac2ef2\" (UID: \"a9280996-adf2-439c-934a-e8fa3aac2ef2\") "
Oct 31 21:36:11.348401 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:11.348369    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9280996-adf2-439c-934a-e8fa3aac2ef2-plugins-config-volume" pod "a9280996-adf2-439c-934a-e8fa3aac2ef2" (UID: "a9280996-adf2-439c-934a-e8fa3aac2ef2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:36:11.349286 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:11.349251    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9280996-adf2-439c-934a-e8fa3aac2ef2-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:36:11.351158 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:11.351130    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9280996-adf2-439c-934a-e8fa3aac2ef2-kube-api-access-8x7wm" pod "a9280996-adf2-439c-934a-e8fa3aac2ef2" (UID: "a9280996-adf2-439c-934a-e8fa3aac2ef2"). InnerVolumeSpecName "kube-api-access-8x7wm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:36:11.361389 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:11.361359    4460 generic.go:356] "Generic (PLEG): container finished" podID="a9280996-adf2-439c-934a-e8fa3aac2ef2" containerID="7773354f12bb59f9635d17d24402abb89d6306670563ee788362db1ee9ea127c" exitCode=0
Oct 31 21:36:11.361616 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:11.361404    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-248b29-7c8fd468f5-fm992" event={"ID":"a9280996-adf2-439c-934a-e8fa3aac2ef2","Type":"ContainerDied","Data":"7773354f12bb59f9635d17d24402abb89d6306670563ee788362db1ee9ea127c"}
Oct 31 21:36:11.361720 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:11.361656    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-248b29-7c8fd468f5-fm992" event={"ID":"a9280996-adf2-439c-934a-e8fa3aac2ef2","Type":"ContainerDied","Data":"cd78a7051bee3f34ac0e7006fe6f53f605402d8a730ff3b25048e844c3570adb"}
Oct 31 21:36:11.361720 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:11.361418    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-248b29-7c8fd468f5-fm992"
Oct 31 21:36:11.361720 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:11.361698    4460 scope.go:122] "RemoveContainer" containerID="7773354f12bb59f9635d17d24402abb89d6306670563ee788362db1ee9ea127c"
Oct 31 21:36:11.377663 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:11.377640    4460 scope.go:122] "RemoveContainer" containerID="7773354f12bb59f9635d17d24402abb89d6306670563ee788362db1ee9ea127c"
Oct 31 21:36:11.378116 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:36:11.378091    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7773354f12bb59f9635d17d24402abb89d6306670563ee788362db1ee9ea127c\": container with ID starting with 7773354f12bb59f9635d17d24402abb89d6306670563ee788362db1ee9ea127c not found: ID does not exist" containerID="7773354f12bb59f9635d17d24402abb89d6306670563ee788362db1ee9ea127c"
Oct 31 21:36:11.378234 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:11.378120    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7773354f12bb59f9635d17d24402abb89d6306670563ee788362db1ee9ea127c"} err="failed to get container status \"7773354f12bb59f9635d17d24402abb89d6306670563ee788362db1ee9ea127c\": rpc error: code = NotFound desc = could not find container \"7773354f12bb59f9635d17d24402abb89d6306670563ee788362db1ee9ea127c\": container with ID starting with 7773354f12bb59f9635d17d24402abb89d6306670563ee788362db1ee9ea127c not found: ID does not exist"
Oct 31 21:36:11.406333 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:11.406305    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:11.420097 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:11.420075    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-248b29-7c8fd468f5-fm992"]
Oct 31 21:36:11.423934 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:11.423914    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-248b29-7c8fd468f5-fm992"]
Oct 31 21:36:11.466176 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:11.451544    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8x7wm\" (UniqueName: \"kubernetes.io/projected/a9280996-adf2-439c-934a-e8fa3aac2ef2-kube-api-access-8x7wm\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:36:11.565964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:11.565356    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:11.687637 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:11.687064    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:11.869254 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:11.869160    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"]
Oct 31 21:36:11.869480 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:11.869452    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr" podUID="8842af32-490d-40d9-9da3-e9daba253252" containerName="istio-proxy" containerID="cri-o://19d115c4f1afe5fa2f1c4cdd1b0fa3605dcf3a98e5fefc7dd058be1f05bb1eee" gracePeriod=30
Oct 31 21:36:11.906181 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:11.906149    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:12.251016 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:12.248684    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a9280996-adf2-439c-934a-e8fa3aac2ef2" path="/var/lib/kubelet/pods/a9280996-adf2-439c-934a-e8fa3aac2ef2/volumes"
Oct 31 21:36:13.186328 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:13.186252    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:14.183014 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:14.161927    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:36:15.148070 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:15.148036    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:15.535970 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:15.535896    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:15.644581 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:15.644544    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:16.457052 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:16.457003    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:16.589194 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:16.589160    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:17.205047 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.203208    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:17.440966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.437782    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:36:17.451704 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.451671    4460 generic.go:356] "Generic (PLEG): container finished" podID="8842af32-490d-40d9-9da3-e9daba253252" containerID="19d115c4f1afe5fa2f1c4cdd1b0fa3605dcf3a98e5fefc7dd058be1f05bb1eee" exitCode=0
Oct 31 21:36:17.451889 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.451716    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"
Oct 31 21:36:17.451889 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.451758    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr" event={"ID":"8842af32-490d-40d9-9da3-e9daba253252","Type":"ContainerDied","Data":"19d115c4f1afe5fa2f1c4cdd1b0fa3605dcf3a98e5fefc7dd058be1f05bb1eee"}
Oct 31 21:36:17.451889 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.451815    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr" event={"ID":"8842af32-490d-40d9-9da3-e9daba253252","Type":"ContainerDied","Data":"42e97361970898880c3b09e7d5a7808e1548b3c584b98b5c4b3827e51204beb2"}
Oct 31 21:36:17.451889 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.451843    4460 scope.go:122] "RemoveContainer" containerID="19d115c4f1afe5fa2f1c4cdd1b0fa3605dcf3a98e5fefc7dd058be1f05bb1eee"
Oct 31 21:36:17.462231 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.462198    4460 scope.go:122] "RemoveContainer" containerID="939d3519c52c12e7c628467a6e348dbbfd3ec477c78d265f11274b60d5cc3919"
Oct 31 21:36:17.471542 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.471525    4460 scope.go:122] "RemoveContainer" containerID="19d115c4f1afe5fa2f1c4cdd1b0fa3605dcf3a98e5fefc7dd058be1f05bb1eee"
Oct 31 21:36:17.471806 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:36:17.471787    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19d115c4f1afe5fa2f1c4cdd1b0fa3605dcf3a98e5fefc7dd058be1f05bb1eee\": container with ID starting with 19d115c4f1afe5fa2f1c4cdd1b0fa3605dcf3a98e5fefc7dd058be1f05bb1eee not found: ID does not exist" containerID="19d115c4f1afe5fa2f1c4cdd1b0fa3605dcf3a98e5fefc7dd058be1f05bb1eee"
Oct 31 21:36:17.471859 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.471816    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19d115c4f1afe5fa2f1c4cdd1b0fa3605dcf3a98e5fefc7dd058be1f05bb1eee"} err="failed to get container status \"19d115c4f1afe5fa2f1c4cdd1b0fa3605dcf3a98e5fefc7dd058be1f05bb1eee\": rpc error: code = NotFound desc = could not find container \"19d115c4f1afe5fa2f1c4cdd1b0fa3605dcf3a98e5fefc7dd058be1f05bb1eee\": container with ID starting with 19d115c4f1afe5fa2f1c4cdd1b0fa3605dcf3a98e5fefc7dd058be1f05bb1eee not found: ID does not exist"
Oct 31 21:36:17.471859 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.471832    4460 scope.go:122] "RemoveContainer" containerID="939d3519c52c12e7c628467a6e348dbbfd3ec477c78d265f11274b60d5cc3919"
Oct 31 21:36:17.472116 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:36:17.472088    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"939d3519c52c12e7c628467a6e348dbbfd3ec477c78d265f11274b60d5cc3919\": container with ID starting with 939d3519c52c12e7c628467a6e348dbbfd3ec477c78d265f11274b60d5cc3919 not found: ID does not exist" containerID="939d3519c52c12e7c628467a6e348dbbfd3ec477c78d265f11274b60d5cc3919"
Oct 31 21:36:17.472184 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.472126    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"939d3519c52c12e7c628467a6e348dbbfd3ec477c78d265f11274b60d5cc3919"} err="failed to get container status \"939d3519c52c12e7c628467a6e348dbbfd3ec477c78d265f11274b60d5cc3919\": rpc error: code = NotFound desc = could not find container \"939d3519c52c12e7c628467a6e348dbbfd3ec477c78d265f11274b60d5cc3919\": container with ID starting with 939d3519c52c12e7c628467a6e348dbbfd3ec477c78d265f11274b60d5cc3919 not found: ID does not exist"
Oct 31 21:36:17.500048 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.500028    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8842af32-490d-40d9-9da3-e9daba253252-kube-api-access-9b5dg\" (UniqueName: \"kubernetes.io/projected/8842af32-490d-40d9-9da3-e9daba253252-kube-api-access-9b5dg\") pod \"8842af32-490d-40d9-9da3-e9daba253252\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") "
Oct 31 21:36:17.500145 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.500124    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-credential-socket\") pod \"8842af32-490d-40d9-9da3-e9daba253252\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") "
Oct 31 21:36:17.500215 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.500163    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8842af32-490d-40d9-9da3-e9daba253252-istio-token\" (UniqueName: \"kubernetes.io/projected/8842af32-490d-40d9-9da3-e9daba253252-istio-token\") pod \"8842af32-490d-40d9-9da3-e9daba253252\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") "
Oct 31 21:36:17.500215 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.500189    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8842af32-490d-40d9-9da3-e9daba253252-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8842af32-490d-40d9-9da3-e9daba253252-istio-podinfo\") pod \"8842af32-490d-40d9-9da3-e9daba253252\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") "
Oct 31 21:36:17.500324 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.500252    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8842af32-490d-40d9-9da3-e9daba253252-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8842af32-490d-40d9-9da3-e9daba253252-istiod-ca-cert\") pod \"8842af32-490d-40d9-9da3-e9daba253252\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") "
Oct 31 21:36:17.500904 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.500403    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-istio-envoy\") pod \"8842af32-490d-40d9-9da3-e9daba253252\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") "
Oct 31 21:36:17.500904 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.500440    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-credential-socket" pod "8842af32-490d-40d9-9da3-e9daba253252" (UID: "8842af32-490d-40d9-9da3-e9daba253252"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:36:17.500904 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.500544    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-istio-data\") pod \"8842af32-490d-40d9-9da3-e9daba253252\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") "
Oct 31 21:36:17.500904 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.500606    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-workload-socket\") pod \"8842af32-490d-40d9-9da3-e9daba253252\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") "
Oct 31 21:36:17.500904 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.500625    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8842af32-490d-40d9-9da3-e9daba253252-istiod-ca-cert" pod "8842af32-490d-40d9-9da3-e9daba253252" (UID: "8842af32-490d-40d9-9da3-e9daba253252"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:36:17.500904 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.500704    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-workload-certs\") pod \"8842af32-490d-40d9-9da3-e9daba253252\" (UID: \"8842af32-490d-40d9-9da3-e9daba253252\") "
Oct 31 21:36:17.500904 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.500841    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-istio-data" pod "8842af32-490d-40d9-9da3-e9daba253252" (UID: "8842af32-490d-40d9-9da3-e9daba253252"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:36:17.500904 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.500887    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-workload-socket" pod "8842af32-490d-40d9-9da3-e9daba253252" (UID: "8842af32-490d-40d9-9da3-e9daba253252"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:36:17.501406 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.501125    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-workload-certs" pod "8842af32-490d-40d9-9da3-e9daba253252" (UID: "8842af32-490d-40d9-9da3-e9daba253252"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:36:17.501493 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.501468    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:36:17.501592 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.501498    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:36:17.501592 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.501510    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:36:17.501592 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.501518    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8842af32-490d-40d9-9da3-e9daba253252-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:36:17.501592 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.501527    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:36:17.502679 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.502658    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8842af32-490d-40d9-9da3-e9daba253252-istio-podinfo" pod "8842af32-490d-40d9-9da3-e9daba253252" (UID: "8842af32-490d-40d9-9da3-e9daba253252"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:36:17.502762 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.502675    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8842af32-490d-40d9-9da3-e9daba253252-kube-api-access-9b5dg" pod "8842af32-490d-40d9-9da3-e9daba253252" (UID: "8842af32-490d-40d9-9da3-e9daba253252"). InnerVolumeSpecName "kube-api-access-9b5dg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:36:17.502804 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.502783    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-istio-envoy" pod "8842af32-490d-40d9-9da3-e9daba253252" (UID: "8842af32-490d-40d9-9da3-e9daba253252"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:36:17.502804 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.502792    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8842af32-490d-40d9-9da3-e9daba253252-istio-token" pod "8842af32-490d-40d9-9da3-e9daba253252" (UID: "8842af32-490d-40d9-9da3-e9daba253252"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:36:17.602996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.602964    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8842af32-490d-40d9-9da3-e9daba253252-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:36:17.602996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.602993    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8842af32-490d-40d9-9da3-e9daba253252-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:36:17.602996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.603002    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8842af32-490d-40d9-9da3-e9daba253252-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:36:17.603445 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.603013    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9b5dg\" (UniqueName: \"kubernetes.io/projected/8842af32-490d-40d9-9da3-e9daba253252-kube-api-access-9b5dg\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:36:17.802967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.802921    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"]
Oct 31 21:36:17.816498 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:17.816473    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-371531-openshift-default-d977d868c-nzsxr"]
Oct 31 21:36:18.250575 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:18.250490    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8842af32-490d-40d9-9da3-e9daba253252" path="/var/lib/kubelet/pods/8842af32-490d-40d9-9da3-e9daba253252/volumes"
Oct 31 21:36:18.363071 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:18.363033    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:19.084589 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:19.084551    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:19.242011 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:19.241972    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:19.517255 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:19.517224    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:19.638385 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:19.638351    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:20.935180 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:20.935145    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:21.490083 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:36:21.490048    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:36:22.720850 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:22.720815    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:24.499973 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:24.499918    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:24.971916 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:24.971875    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:26.204712 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:26.204656    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:27.551772 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:27.551738    4460 generic.go:356] "Generic (PLEG): container finished" podID="dd82789d-81ca-487e-8bfb-e2567ca866bd" containerID="dc44e026b70ed551abcbbbc17389ec363ff129c85ceeb95bb992464be87eabdb" exitCode=0
Oct 31 21:36:27.552236 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:27.551812    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs" event={"ID":"dd82789d-81ca-487e-8bfb-e2567ca866bd","Type":"ContainerDied","Data":"dc44e026b70ed551abcbbbc17389ec363ff129c85ceeb95bb992464be87eabdb"}
Oct 31 21:36:27.552284 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:27.552246    4460 scope.go:122] "RemoveContainer" containerID="dc44e026b70ed551abcbbbc17389ec363ff129c85ceeb95bb992464be87eabdb"
Oct 31 21:36:27.553616 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:27.553587    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:36:27.553678 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:27.553655    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:36:27.553717 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:27.553686    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:36:27.658135 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:27.658101    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:27.784067 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:27.784038    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:36:28.177964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:28.174397    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:28.337960 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:28.337654    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:28.454034 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:28.453921    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:28.616014 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:28.615076    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs" event={"ID":"dd82789d-81ca-487e-8bfb-e2567ca866bd","Type":"ContainerStarted","Data":"0ad93cdd8a1238afcb38b58a313cfc4496716b90a56093f52b7d6cb54a832cf5"}
Oct 31 21:36:28.846989 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:28.842383    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:29.174341 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:29.174256    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:36:29.175754 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:29.175727    4460 patch_prober.go:29] interesting pod/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 31 21:36:29.175876 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:29.175786    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs" podUID="dd82789d-81ca-487e-8bfb-e2567ca866bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 31 21:36:30.174970 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:30.174926    4460 patch_prober.go:29] interesting pod/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 31 21:36:30.175357 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:30.174991    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs" podUID="dd82789d-81ca-487e-8bfb-e2567ca866bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 31 21:36:30.368274 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:30.368239    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:31.175362 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:31.175325    4460 patch_prober.go:29] interesting pod/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 31 21:36:31.176121 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:31.176080    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs" podUID="dd82789d-81ca-487e-8bfb-e2567ca866bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 31 21:36:32.174529 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:32.174490    4460 patch_prober.go:29] interesting pod/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 31 21:36:32.174740 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:32.174594    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs" podUID="dd82789d-81ca-487e-8bfb-e2567ca866bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 31 21:36:32.318431 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:32.318384    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:32.446370 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:32.446276    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-08ddf0-54474ff8d7-vxf9n"]
Oct 31 21:36:32.555975 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:32.555441    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:32.669817 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:32.669320    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:33.175462 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:33.175423    4460 patch_prober.go:29] interesting pod/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 31 21:36:33.175773 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:33.175743    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs" podUID="dd82789d-81ca-487e-8bfb-e2567ca866bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 31 21:36:34.104465 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:34.104431    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:34.177340 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:34.177300    4460 patch_prober.go:29] interesting pod/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Oct 31 21:36:34.177659 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:34.177622    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs" podUID="dd82789d-81ca-487e-8bfb-e2567ca866bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Oct 31 21:36:35.179318 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:35.179284    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:36:35.179710 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:35.179691    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:36:35.180336 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:35.180316    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:36:35.921450 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:35.921410    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:36.051997 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:36.048437    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:36.333551 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:36.333512    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"]
Oct 31 21:36:36.334101 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:36.333770    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj" podUID="8cfc07f9-3f4a-445d-9880-a7b25a95b3dd" containerName="istio-proxy" containerID="cri-o://216125795b7ad5da329a61eccb02690b327e33a376ff35edca312388f6fe88f4" gracePeriod=30
Oct 31 21:36:36.369368 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:36.369315    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:37.454137 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:37.454074    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-08ddf0-54474ff8d7-vxf9n" podUID="4ca7b94b-2aef-45f5-8551-7ebfd685fe4b" containerName="payload-pre-processing" containerID="cri-o://4b7c86eaef4b71dcefbe923a2395f6bf2b650dc564c5a4ff9f861d15ff0dd689" gracePeriod=25
Oct 31 21:36:37.802026 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:37.802002    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-08ddf0-54474ff8d7-vxf9n"
Oct 31 21:36:37.842980 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:37.842919    4460 generic.go:356] "Generic (PLEG): container finished" podID="4ca7b94b-2aef-45f5-8551-7ebfd685fe4b" containerID="4b7c86eaef4b71dcefbe923a2395f6bf2b650dc564c5a4ff9f861d15ff0dd689" exitCode=0
Oct 31 21:36:37.843153 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:37.842973    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-08ddf0-54474ff8d7-vxf9n" event={"ID":"4ca7b94b-2aef-45f5-8551-7ebfd685fe4b","Type":"ContainerDied","Data":"4b7c86eaef4b71dcefbe923a2395f6bf2b650dc564c5a4ff9f861d15ff0dd689"}
Oct 31 21:36:37.843153 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:37.843035    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-08ddf0-54474ff8d7-vxf9n" event={"ID":"4ca7b94b-2aef-45f5-8551-7ebfd685fe4b","Type":"ContainerDied","Data":"fa2cf33d30517994e98777f73afc455b328425269ee4c2a50237df0934f34846"}
Oct 31 21:36:37.843153 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:37.843060    4460 scope.go:122] "RemoveContainer" containerID="4b7c86eaef4b71dcefbe923a2395f6bf2b650dc564c5a4ff9f861d15ff0dd689"
Oct 31 21:36:37.843153 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:37.842980    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-08ddf0-54474ff8d7-vxf9n"
Oct 31 21:36:37.854035 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:37.854006    4460 scope.go:122] "RemoveContainer" containerID="4b7c86eaef4b71dcefbe923a2395f6bf2b650dc564c5a4ff9f861d15ff0dd689"
Oct 31 21:36:37.854342 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:36:37.854313    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b7c86eaef4b71dcefbe923a2395f6bf2b650dc564c5a4ff9f861d15ff0dd689\": container with ID starting with 4b7c86eaef4b71dcefbe923a2395f6bf2b650dc564c5a4ff9f861d15ff0dd689 not found: ID does not exist" containerID="4b7c86eaef4b71dcefbe923a2395f6bf2b650dc564c5a4ff9f861d15ff0dd689"
Oct 31 21:36:37.854415 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:37.854347    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b7c86eaef4b71dcefbe923a2395f6bf2b650dc564c5a4ff9f861d15ff0dd689"} err="failed to get container status \"4b7c86eaef4b71dcefbe923a2395f6bf2b650dc564c5a4ff9f861d15ff0dd689\": rpc error: code = NotFound desc = could not find container \"4b7c86eaef4b71dcefbe923a2395f6bf2b650dc564c5a4ff9f861d15ff0dd689\": container with ID starting with 4b7c86eaef4b71dcefbe923a2395f6bf2b650dc564c5a4ff9f861d15ff0dd689 not found: ID does not exist"
Oct 31 21:36:37.880362 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:37.880328    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:37.981963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:37.979970    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4ca7b94b-2aef-45f5-8551-7ebfd685fe4b-kube-api-access-q25n4\" (UniqueName: \"kubernetes.io/projected/4ca7b94b-2aef-45f5-8551-7ebfd685fe4b-kube-api-access-q25n4\") pod \"4ca7b94b-2aef-45f5-8551-7ebfd685fe4b\" (UID: \"4ca7b94b-2aef-45f5-8551-7ebfd685fe4b\") "
Oct 31 21:36:37.981963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:37.980261    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4ca7b94b-2aef-45f5-8551-7ebfd685fe4b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ca7b94b-2aef-45f5-8551-7ebfd685fe4b-plugins-config-volume\") pod \"4ca7b94b-2aef-45f5-8551-7ebfd685fe4b\" (UID: \"4ca7b94b-2aef-45f5-8551-7ebfd685fe4b\") "
Oct 31 21:36:37.981963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:37.981520    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ca7b94b-2aef-45f5-8551-7ebfd685fe4b-plugins-config-volume" pod "4ca7b94b-2aef-45f5-8551-7ebfd685fe4b" (UID: "4ca7b94b-2aef-45f5-8551-7ebfd685fe4b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:36:38.010922 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:38.010882    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ca7b94b-2aef-45f5-8551-7ebfd685fe4b-kube-api-access-q25n4" pod "4ca7b94b-2aef-45f5-8551-7ebfd685fe4b" (UID: "4ca7b94b-2aef-45f5-8551-7ebfd685fe4b"). InnerVolumeSpecName "kube-api-access-q25n4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:36:38.082925 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:38.082856    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ca7b94b-2aef-45f5-8551-7ebfd685fe4b-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:36:38.082925 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:38.082883    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q25n4\" (UniqueName: \"kubernetes.io/projected/4ca7b94b-2aef-45f5-8551-7ebfd685fe4b-kube-api-access-q25n4\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:36:38.193301 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:38.193268    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-08ddf0-54474ff8d7-vxf9n"]
Oct 31 21:36:38.207099 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:38.207074    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-08ddf0-54474ff8d7-vxf9n"]
Oct 31 21:36:38.226403 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:38.226375    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4ca7b94b-2aef-45f5-8551-7ebfd685fe4b" path="/var/lib/kubelet/pods/4ca7b94b-2aef-45f5-8551-7ebfd685fe4b/volumes"
Oct 31 21:36:39.717159 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:39.717118    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:40.589915 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:40.589876    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:41.038686 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.038653    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"]
Oct 31 21:36:41.039657 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.039631    4460 cpu_manager.go:414] "containerMap: removing container" podUID="8842af32-490d-40d9-9da3-e9daba253252" containerName="istio-proxy"
Oct 31 21:36:41.039657 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.039655    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8842af32-490d-40d9-9da3-e9daba253252" containerName="istio-proxy"
Oct 31 21:36:41.039795 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.039683    4460 cpu_manager.go:414] "containerMap: removing container" podUID="a9280996-adf2-439c-934a-e8fa3aac2ef2" containerName="payload-processing"
Oct 31 21:36:41.039795 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.039688    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9280996-adf2-439c-934a-e8fa3aac2ef2" containerName="payload-processing"
Oct 31 21:36:41.039795 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.039702    4460 cpu_manager.go:414] "containerMap: removing container" podUID="4ca7b94b-2aef-45f5-8551-7ebfd685fe4b" containerName="payload-pre-processing"
Oct 31 21:36:41.039795 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.039708    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4ca7b94b-2aef-45f5-8551-7ebfd685fe4b" containerName="payload-pre-processing"
Oct 31 21:36:41.039919 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.039817    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="8842af32-490d-40d9-9da3-e9daba253252" containerName="istio-proxy"
Oct 31 21:36:41.039919 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.039828    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9280996-adf2-439c-934a-e8fa3aac2ef2" containerName="payload-processing"
Oct 31 21:36:41.039919 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.039835    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="4ca7b94b-2aef-45f5-8551-7ebfd685fe4b" containerName="payload-pre-processing"
Oct 31 21:36:41.043434 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.043412    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:36:41.061253 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.061226    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w4-f1e791-openshift-default-dockercfg-5wcx4\""
Oct 31 21:36:41.078819 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.078792    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"]
Oct 31 21:36:41.106409 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.106369    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:41.126070 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.126041    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-workload-socket\") pod \"e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") " pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:36:41.126209 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.126090    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istio-token\") pod \"e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") " pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:36:41.126209 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.126115    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istio-envoy\") pod \"e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") " pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:36:41.126209 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.126186    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-credential-socket\") pod \"e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") " pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:36:41.126328 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.126257    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f459v\" (UniqueName: \"kubernetes.io/projected/9a09d750-0cc0-430b-a2d6-6490cf9f1361-kube-api-access-f459v\") pod \"e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") " pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:36:41.126399 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.126386    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istio-data\") pod \"e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") " pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:36:41.126438 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.126423    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-workload-certs\") pod \"e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") " pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:36:41.126471 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.126455    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istiod-ca-cert\") pod \"e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") " pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:36:41.126557 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.126529    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istio-podinfo\") pod \"e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") " pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:36:41.278963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.272781    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istio-envoy\") pod \"e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") " pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:36:41.278963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.272835    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-credential-socket\") pod \"e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") " pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:36:41.278963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.272887    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f459v\" (UniqueName: \"kubernetes.io/projected/9a09d750-0cc0-430b-a2d6-6490cf9f1361-kube-api-access-f459v\") pod \"e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") " pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:36:41.278963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.273066    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istio-data\") pod \"e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") " pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:36:41.278963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.273114    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-workload-certs\") pod \"e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") " pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:36:41.278963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.273159    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istiod-ca-cert\") pod \"e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") " pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:36:41.278963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.273194    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istio-podinfo\") pod \"e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") " pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:36:41.278963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.273321    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-workload-socket\") pod \"e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") " pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:36:41.278963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.273372    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istio-token\") pod \"e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") " pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:36:41.291069 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.283423    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-credential-socket\") pod \"e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") " pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:36:41.291069 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.284001    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istio-data\") pod \"e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") " pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:36:41.291069 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.284829    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istiod-ca-cert\") pod \"e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") " pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:36:41.291069 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.285491    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-workload-socket\") pod \"e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") " pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:36:41.291069 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.285755    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-workload-certs\") pod \"e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") " pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:36:41.333108 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.333027    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istio-podinfo\") pod \"e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") " pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:36:41.345979 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.341134    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:41.347386 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.347318    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istio-envoy\") pod \"e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") " pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:36:41.348061 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.348032    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istio-token\") pod \"e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") " pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:36:41.358058 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.356607    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f459v\" (UniqueName: \"kubernetes.io/projected/9a09d750-0cc0-430b-a2d6-6490cf9f1361-kube-api-access-f459v\") pod \"e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") " pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:36:41.361985 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.358775    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:36:41.747522 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.747498    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:36:41.810460 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.810384    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istiod-ca-cert\") pod \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") "
Oct 31 21:36:41.810623 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.810485    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-credential-socket\") pod \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") "
Oct 31 21:36:41.810623 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.810580    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istio-podinfo\") pod \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") "
Oct 31 21:36:41.810733 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.810706    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istio-envoy\") pod \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") "
Oct 31 21:36:41.810809 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.810790    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-workload-socket\") pod \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") "
Oct 31 21:36:41.810868 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.810830    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-kube-api-access-b5qv7\" (UniqueName: \"kubernetes.io/projected/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-kube-api-access-b5qv7\") pod \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") "
Oct 31 21:36:41.810868 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.810838    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istiod-ca-cert" pod "8cfc07f9-3f4a-445d-9880-a7b25a95b3dd" (UID: "8cfc07f9-3f4a-445d-9880-a7b25a95b3dd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:36:41.810998 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.810887    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istio-token\" (UniqueName: \"kubernetes.io/projected/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istio-token\") pod \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") "
Oct 31 21:36:41.810998 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.810927    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-workload-certs\") pod \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") "
Oct 31 21:36:41.810998 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.810991    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istio-data\") pod \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\" (UID: \"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd\") "
Oct 31 21:36:41.811149 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.811088    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-workload-socket" pod "8cfc07f9-3f4a-445d-9880-a7b25a95b3dd" (UID: "8cfc07f9-3f4a-445d-9880-a7b25a95b3dd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:36:41.811452 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.811429    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-workload-certs" pod "8cfc07f9-3f4a-445d-9880-a7b25a95b3dd" (UID: "8cfc07f9-3f4a-445d-9880-a7b25a95b3dd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:36:41.811648 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.811567    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istio-data" pod "8cfc07f9-3f4a-445d-9880-a7b25a95b3dd" (UID: "8cfc07f9-3f4a-445d-9880-a7b25a95b3dd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:36:41.811916 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.811896    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:36:41.812059 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.812044    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:36:41.812158 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.812147    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:36:41.812258 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.812246    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:36:41.813750 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.813710    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-kube-api-access-b5qv7" pod "8cfc07f9-3f4a-445d-9880-a7b25a95b3dd" (UID: "8cfc07f9-3f4a-445d-9880-a7b25a95b3dd"). InnerVolumeSpecName "kube-api-access-b5qv7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:36:41.815461 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.815437    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istio-token" pod "8cfc07f9-3f4a-445d-9880-a7b25a95b3dd" (UID: "8cfc07f9-3f4a-445d-9880-a7b25a95b3dd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:36:41.815559 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.815523    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istio-podinfo" pod "8cfc07f9-3f4a-445d-9880-a7b25a95b3dd" (UID: "8cfc07f9-3f4a-445d-9880-a7b25a95b3dd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:36:41.815624 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.815577    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-credential-socket" pod "8cfc07f9-3f4a-445d-9880-a7b25a95b3dd" (UID: "8cfc07f9-3f4a-445d-9880-a7b25a95b3dd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:36:41.815780 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.815754    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istio-envoy" pod "8cfc07f9-3f4a-445d-9880-a7b25a95b3dd" (UID: "8cfc07f9-3f4a-445d-9880-a7b25a95b3dd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:36:41.816231 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.816207    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:41.823640 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.823570    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"]
Oct 31 21:36:41.833259 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:36:41.833236    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a09d750_0cc0_430b_a2d6_6490cf9f1361.slice/crio-3d1bd55417f1a096aa31cd8038d8dacdb32756b5f19b5ff1cef1490de44d8762 WatchSource:0}: Error finding container 3d1bd55417f1a096aa31cd8038d8dacdb32756b5f19b5ff1cef1490de44d8762: Status 404 returned error can't find the container with id 3d1bd55417f1a096aa31cd8038d8dacdb32756b5f19b5ff1cef1490de44d8762
Oct 31 21:36:41.835983 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.835932    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:36:41.836061 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.836015    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:36:41.836061 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.836044    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:36:41.912990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.902372    4460 generic.go:356] "Generic (PLEG): container finished" podID="8cfc07f9-3f4a-445d-9880-a7b25a95b3dd" containerID="216125795b7ad5da329a61eccb02690b327e33a376ff35edca312388f6fe88f4" exitCode=0
Oct 31 21:36:41.912990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.902499    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj" event={"ID":"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd","Type":"ContainerDied","Data":"216125795b7ad5da329a61eccb02690b327e33a376ff35edca312388f6fe88f4"}
Oct 31 21:36:41.912990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.902573    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj" event={"ID":"8cfc07f9-3f4a-445d-9880-a7b25a95b3dd","Type":"ContainerDied","Data":"1fdbf904118a0d0936980850e8f480391bd9fdbf436c8e25bf4775df932154e9"}
Oct 31 21:36:41.912990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.902605    4460 scope.go:122] "RemoveContainer" containerID="216125795b7ad5da329a61eccb02690b327e33a376ff35edca312388f6fe88f4"
Oct 31 21:36:41.912990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.902769    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"
Oct 31 21:36:41.939991 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.939225    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:36:41.939991 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.939261    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:36:41.939991 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.939284    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:36:41.939991 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.939301    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5qv7\" (UniqueName: \"kubernetes.io/projected/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-kube-api-access-b5qv7\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:36:41.939991 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.939314    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:36:41.976962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.976070    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm" event={"ID":"9a09d750-0cc0-430b-a2d6-6490cf9f1361","Type":"ContainerStarted","Data":"3d1bd55417f1a096aa31cd8038d8dacdb32756b5f19b5ff1cef1490de44d8762"}
Oct 31 21:36:41.988075 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.987914    4460 scope.go:122] "RemoveContainer" containerID="adbd0fb8e5ff7354007505510805f27a19ac4d6b95f04a8274bbeb79a5ecd254"
Oct 31 21:36:41.997486 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.997456    4460 scope.go:122] "RemoveContainer" containerID="216125795b7ad5da329a61eccb02690b327e33a376ff35edca312388f6fe88f4"
Oct 31 21:36:41.997825 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:36:41.997795    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"216125795b7ad5da329a61eccb02690b327e33a376ff35edca312388f6fe88f4\": container with ID starting with 216125795b7ad5da329a61eccb02690b327e33a376ff35edca312388f6fe88f4 not found: ID does not exist" containerID="216125795b7ad5da329a61eccb02690b327e33a376ff35edca312388f6fe88f4"
Oct 31 21:36:41.997920 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.997839    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"216125795b7ad5da329a61eccb02690b327e33a376ff35edca312388f6fe88f4"} err="failed to get container status \"216125795b7ad5da329a61eccb02690b327e33a376ff35edca312388f6fe88f4\": rpc error: code = NotFound desc = could not find container \"216125795b7ad5da329a61eccb02690b327e33a376ff35edca312388f6fe88f4\": container with ID starting with 216125795b7ad5da329a61eccb02690b327e33a376ff35edca312388f6fe88f4 not found: ID does not exist"
Oct 31 21:36:41.997920 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.997864    4460 scope.go:122] "RemoveContainer" containerID="adbd0fb8e5ff7354007505510805f27a19ac4d6b95f04a8274bbeb79a5ecd254"
Oct 31 21:36:41.998212 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:36:41.998182    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adbd0fb8e5ff7354007505510805f27a19ac4d6b95f04a8274bbeb79a5ecd254\": container with ID starting with adbd0fb8e5ff7354007505510805f27a19ac4d6b95f04a8274bbeb79a5ecd254 not found: ID does not exist" containerID="adbd0fb8e5ff7354007505510805f27a19ac4d6b95f04a8274bbeb79a5ecd254"
Oct 31 21:36:41.998259 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:41.998218    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adbd0fb8e5ff7354007505510805f27a19ac4d6b95f04a8274bbeb79a5ecd254"} err="failed to get container status \"adbd0fb8e5ff7354007505510805f27a19ac4d6b95f04a8274bbeb79a5ecd254\": rpc error: code = NotFound desc = could not find container \"adbd0fb8e5ff7354007505510805f27a19ac4d6b95f04a8274bbeb79a5ecd254\": container with ID starting with adbd0fb8e5ff7354007505510805f27a19ac4d6b95f04a8274bbeb79a5ecd254 not found: ID does not exist"
Oct 31 21:36:42.019301 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:42.019274    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"]
Oct 31 21:36:42.024431 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:42.024398    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-248b29-openshift-default-66f64586d5-94rjj"]
Oct 31 21:36:42.233372 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:42.233286    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8cfc07f9-3f4a-445d-9880-a7b25a95b3dd" path="/var/lib/kubelet/pods/8cfc07f9-3f4a-445d-9880-a7b25a95b3dd/volumes"
Oct 31 21:36:42.985196 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:42.985159    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm" event={"ID":"9a09d750-0cc0-430b-a2d6-6490cf9f1361","Type":"ContainerStarted","Data":"3042a385bef9cf71215d9cac8aa2098c635c2f82effbb7d7b18e08bb1726d6ec"}
Oct 31 21:36:43.052568 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:43.052516    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm" podStartSLOduration=2.05250415 podStartE2EDuration="2.05250415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:36:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:36:43.050441393 +0000 UTC m=+1242.966750826" watchObservedRunningTime="2026-10-31 21:36:43.05250415 +0000 UTC m=+1242.968813646"
Oct 31 21:36:43.359805 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:43.359767    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:36:43.365693 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:43.365664    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:36:43.527270 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:43.527230    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:43.991808 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:43.991769    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:36:43.992738 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:43.992718    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:36:45.387350 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:45.387318    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:45.504442 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:45.504398    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:45.617112 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:45.617078    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:45.980405 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:45.980370    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:47.212428 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:47.212388    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:47.340086 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:47.340046    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:47.462008 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:47.461930    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:49.239155 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:49.239115    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:50.918095 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:50.917914    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:51.454194 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:36:51.454157    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:36:52.607394 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:52.607351    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:52.967535 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:52.967455    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:36:54.430491 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:54.430452    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:56.201254 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:56.201073    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k"]
Oct 31 21:36:56.202307 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:56.202252    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k" podUID="6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8" containerName="maas-api" containerID="cri-o://bb74edf28465cc677ed9989d10111ea68f9f67fada1ecb18bf09667d3a5ecd47" gracePeriod=30
Oct 31 21:36:56.328096 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:56.328057    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:56.509411 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:56.507793    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:56.630925 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:56.630889    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:57.114854 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:57.114819    4460 generic.go:356] "Generic (PLEG): container finished" podID="6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8" containerID="bb74edf28465cc677ed9989d10111ea68f9f67fada1ecb18bf09667d3a5ecd47" exitCode=0
Oct 31 21:36:57.115164 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:57.115142    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k" event={"ID":"6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8","Type":"ContainerDied","Data":"bb74edf28465cc677ed9989d10111ea68f9f67fada1ecb18bf09667d3a5ecd47"}
Oct 31 21:36:57.115331 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:57.115313    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k" event={"ID":"6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8","Type":"ContainerDied","Data":"4fe2c9c5de72aafbc8dce1b4c9ff5844f9bac3f22e9386777c5ac5b8b7145ef2"}
Oct 31 21:36:57.115450 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:57.115434    4460 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4fe2c9c5de72aafbc8dce1b4c9ff5844f9bac3f22e9386777c5ac5b8b7145ef2"
Oct 31 21:36:57.205084 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:57.205051    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k"
Oct 31 21:36:57.300767 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:57.268488    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:57.382157 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:57.380799    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8-maas-api-tls\") pod \"6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8\" (UID: \"6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8\") "
Oct 31 21:36:57.382157 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:57.380973    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8-kube-api-access-rpl69\" (UniqueName: \"kubernetes.io/projected/6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8-kube-api-access-rpl69\") pod \"6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8\" (UID: \"6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8\") "
Oct 31 21:36:57.382157 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:57.381026    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8-openshift-service-ca\") pod \"6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8\" (UID: \"6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8\") "
Oct 31 21:36:57.382157 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:57.381159    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8-metrics-certs\") pod \"6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8\" (UID: \"6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8\") "
Oct 31 21:36:57.390098 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:57.390050    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8-openshift-service-ca" pod "6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8" (UID: "6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:36:57.391983 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:57.391932    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8-metrics-certs" pod "6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8" (UID: "6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 21:36:57.434868 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:57.408224    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8-kube-api-access-rpl69" pod "6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8" (UID: "6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8"). InnerVolumeSpecName "kube-api-access-rpl69". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:36:57.441215 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:57.441145    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8-maas-api-tls" pod "6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8" (UID: "6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 21:36:57.493985 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:57.490281    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpl69\" (UniqueName: \"kubernetes.io/projected/6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8-kube-api-access-rpl69\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:36:57.493985 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:57.490319    4460 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8-openshift-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:36:57.493985 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:57.490337    4460 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8-metrics-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:36:57.493985 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:57.490350    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8-maas-api-tls\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:36:57.579508 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:57.579473    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:58.121203 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:58.121171    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k"
Oct 31 21:36:58.182887 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:58.182855    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k"]
Oct 31 21:36:58.190282 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:58.190255    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-08ddf0-7d8bb9f898-kt58k"]
Oct 31 21:36:58.210554 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:58.210518    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:36:58.227262 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:58.227235    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8" path="/var/lib/kubelet/pods/6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8/volumes"
Oct 31 21:36:58.410847 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:36:58.410752    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:00.026493 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:00.026458    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:01.684050 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:01.684017    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45"]
Oct 31 21:37:01.684663 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:01.684647    4460 cpu_manager.go:414] "containerMap: removing container" podUID="6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8" containerName="maas-api"
Oct 31 21:37:01.684663 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:01.684663    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8" containerName="maas-api"
Oct 31 21:37:01.684752 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:01.684688    4460 cpu_manager.go:414] "containerMap: removing container" podUID="8842af32-490d-40d9-9da3-e9daba253252" containerName="istio-proxy"
Oct 31 21:37:01.684752 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:01.684694    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8842af32-490d-40d9-9da3-e9daba253252" containerName="istio-proxy"
Oct 31 21:37:01.684752 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:01.684710    4460 cpu_manager.go:414] "containerMap: removing container" podUID="8cfc07f9-3f4a-445d-9880-a7b25a95b3dd" containerName="istio-proxy"
Oct 31 21:37:01.684752 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:01.684715    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8cfc07f9-3f4a-445d-9880-a7b25a95b3dd" containerName="istio-proxy"
Oct 31 21:37:01.684752 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:01.684732    4460 cpu_manager.go:414] "containerMap: removing container" podUID="8cfc07f9-3f4a-445d-9880-a7b25a95b3dd" containerName="istio-proxy"
Oct 31 21:37:01.684752 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:01.684737    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8cfc07f9-3f4a-445d-9880-a7b25a95b3dd" containerName="istio-proxy"
Oct 31 21:37:01.684923 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:01.684827    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="6eeb4fb9-8e89-45d5-a3ee-289ee7ad9bc8" containerName="maas-api"
Oct 31 21:37:01.684923 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:01.684838    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="8842af32-490d-40d9-9da3-e9daba253252" containerName="istio-proxy"
Oct 31 21:37:01.684923 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:01.684846    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="8cfc07f9-3f4a-445d-9880-a7b25a95b3dd" containerName="istio-proxy"
Oct 31 21:37:01.684923 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:01.684854    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="8cfc07f9-3f4a-445d-9880-a7b25a95b3dd" containerName="istio-proxy"
Oct 31 21:37:01.689656 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:01.689635    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45"
Oct 31 21:37:01.711609 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:01.711580    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45"]
Oct 31 21:37:01.719380 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:01.719354    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w4-f1e791-serving-cert\""
Oct 31 21:37:01.756420 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:01.756377    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93b36b86-cb3d-41a0-b3b8-526a57e417da-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45\" (UID: \"93b36b86-cb3d-41a0-b3b8-526a57e417da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45"
Oct 31 21:37:01.756608 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:01.756503    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhv9p\" (UniqueName: \"kubernetes.io/projected/93b36b86-cb3d-41a0-b3b8-526a57e417da-kube-api-access-qhv9p\") pod \"maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45\" (UID: \"93b36b86-cb3d-41a0-b3b8-526a57e417da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45"
Oct 31 21:37:01.756689 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:01.756618    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93b36b86-cb3d-41a0-b3b8-526a57e417da-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45\" (UID: \"93b36b86-cb3d-41a0-b3b8-526a57e417da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45"
Oct 31 21:37:01.756885 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:01.756862    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93b36b86-cb3d-41a0-b3b8-526a57e417da-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45\" (UID: \"93b36b86-cb3d-41a0-b3b8-526a57e417da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45"
Oct 31 21:37:01.772080 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:01.772013    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:01.858753 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:01.858717    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93b36b86-cb3d-41a0-b3b8-526a57e417da-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45\" (UID: \"93b36b86-cb3d-41a0-b3b8-526a57e417da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45"
Oct 31 21:37:01.858928 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:01.858850    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93b36b86-cb3d-41a0-b3b8-526a57e417da-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45\" (UID: \"93b36b86-cb3d-41a0-b3b8-526a57e417da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45"
Oct 31 21:37:01.858928 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:01.858882    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93b36b86-cb3d-41a0-b3b8-526a57e417da-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45\" (UID: \"93b36b86-cb3d-41a0-b3b8-526a57e417da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45"
Oct 31 21:37:01.858928 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:01.858921    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhv9p\" (UniqueName: \"kubernetes.io/projected/93b36b86-cb3d-41a0-b3b8-526a57e417da-kube-api-access-qhv9p\") pod \"maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45\" (UID: \"93b36b86-cb3d-41a0-b3b8-526a57e417da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45"
Oct 31 21:37:01.859484 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:01.859455    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93b36b86-cb3d-41a0-b3b8-526a57e417da-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45\" (UID: \"93b36b86-cb3d-41a0-b3b8-526a57e417da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45"
Oct 31 21:37:01.861205 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:01.861181    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93b36b86-cb3d-41a0-b3b8-526a57e417da-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45\" (UID: \"93b36b86-cb3d-41a0-b3b8-526a57e417da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45"
Oct 31 21:37:01.861301 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:01.861193    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93b36b86-cb3d-41a0-b3b8-526a57e417da-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45\" (UID: \"93b36b86-cb3d-41a0-b3b8-526a57e417da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45"
Oct 31 21:37:01.916970 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:01.910068    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:01.942009 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:01.940772    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhv9p\" (UniqueName: \"kubernetes.io/projected/93b36b86-cb3d-41a0-b3b8-526a57e417da-kube-api-access-qhv9p\") pod \"maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45\" (UID: \"93b36b86-cb3d-41a0-b3b8-526a57e417da\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45"
Oct 31 21:37:02.010482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:02.010210    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45"
Oct 31 21:37:02.347305 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:02.347279    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45"]
Oct 31 21:37:02.347858 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:37:02.347824    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93b36b86_cb3d_41a0_b3b8_526a57e417da.slice/crio-34fe52054f069652ba0096935317f530463c822d42849d9b11006f4791bc20aa WatchSource:0}: Error finding container 34fe52054f069652ba0096935317f530463c822d42849d9b11006f4791bc20aa: Status 404 returned error can't find the container with id 34fe52054f069652ba0096935317f530463c822d42849d9b11006f4791bc20aa
Oct 31 21:37:03.185280 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:03.185227    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45" event={"ID":"93b36b86-cb3d-41a0-b3b8-526a57e417da","Type":"ContainerStarted","Data":"87d7dde05aa05bd81e466f13fc74917eb0d0b9b85f695ce735e8a0a2834f3bea"}
Oct 31 21:37:03.185675 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:03.185331    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45" event={"ID":"93b36b86-cb3d-41a0-b3b8-526a57e417da","Type":"ContainerStarted","Data":"34fe52054f069652ba0096935317f530463c822d42849d9b11006f4791bc20aa"}
Oct 31 21:37:03.185675 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:03.185364    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45"
Oct 31 21:37:03.464708 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:03.464668    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:04.895610 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:04.895575    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:05.096441 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:05.096398    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:05.209018 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:05.207505    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45" podStartSLOduration=3.839510182 podStartE2EDuration="4.20748036s" totalImagesPullingTime="367.970178ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:37:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:37:03.234705996 +0000 UTC m=+1263.151015425" watchObservedRunningTime="2026-10-31 21:37:05.20748036 +0000 UTC m=+1265.123789791"
Oct 31 21:37:05.285372 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:05.285315    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"]
Oct 31 21:37:05.338869 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:05.338666    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z" podUID="485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5" containerName="istio-proxy" containerID="cri-o://9a712bedde325120c9746e740297a44c2df1db8e77e85dc6e2a463e8736a2782" gracePeriod=30
Oct 31 21:37:05.348261 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:05.348139    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:05.502884 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:05.502787    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:07.140375 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:07.140331    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:08.733957 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:08.733915    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:09.202007 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:09.201969    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45"
Oct 31 21:37:09.293620 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:09.293516    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:09.505308 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:09.505219    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:09.633382 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:09.633346    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:10.012509 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:10.012474    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:10.958714 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:10.958680    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:37:11.064057 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.063980    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-kube-api-access-pd8cc\" (UniqueName: \"kubernetes.io/projected/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-kube-api-access-pd8cc\") pod \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") "
Oct 31 21:37:11.064057 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.064027    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istio-envoy\") pod \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") "
Oct 31 21:37:11.064538 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.064133    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-workload-socket\") pod \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") "
Oct 31 21:37:11.064538 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.064178    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istiod-ca-cert\") pod \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") "
Oct 31 21:37:11.064538 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.064262    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-credential-socket\") pod \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") "
Oct 31 21:37:11.064538 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.064322    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istio-podinfo\") pod \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") "
Oct 31 21:37:11.064538 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.064476    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-workload-socket" pod "485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5" (UID: "485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:37:11.064538 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.064498    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-workload-certs\") pod \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") "
Oct 31 21:37:11.064843 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.064552    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istio-data\") pod \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") "
Oct 31 21:37:11.064843 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.064611    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istio-token\" (UniqueName: \"kubernetes.io/projected/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istio-token\") pod \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\" (UID: \"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5\") "
Oct 31 21:37:11.064995 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.064606    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istiod-ca-cert" pod "485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5" (UID: "485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:37:11.064995 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.064853    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-credential-socket" pod "485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5" (UID: "485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:37:11.065109 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.064910    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-workload-certs" pod "485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5" (UID: "485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:37:11.065525 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.065236    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istio-data" pod "485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5" (UID: "485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:37:11.065525 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.065441    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:37:11.065525 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.065461    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:37:11.065525 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.065473    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:37:11.065525 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.065486    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:37:11.065525 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.065500    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:37:11.067365 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.067339    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istio-podinfo" pod "485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5" (UID: "485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:37:11.067464 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.067346    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istio-envoy" pod "485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5" (UID: "485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:37:11.067464 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.067399    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-kube-api-access-pd8cc" pod "485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5" (UID: "485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5"). InnerVolumeSpecName "kube-api-access-pd8cc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:37:11.067464 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.067428    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istio-token" pod "485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5" (UID: "485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:37:11.166520 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.166492    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:37:11.166520 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.166519    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:37:11.166674 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.166554    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:37:11.166674 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.166564    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pd8cc\" (UniqueName: \"kubernetes.io/projected/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5-kube-api-access-pd8cc\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:37:11.389725 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.389638    4460 generic.go:356] "Generic (PLEG): container finished" podID="485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5" containerID="9a712bedde325120c9746e740297a44c2df1db8e77e85dc6e2a463e8736a2782" exitCode=0
Oct 31 21:37:11.389725 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.389670    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z" event={"ID":"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5","Type":"ContainerDied","Data":"9a712bedde325120c9746e740297a44c2df1db8e77e85dc6e2a463e8736a2782"}
Oct 31 21:37:11.389725 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.389707    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"
Oct 31 21:37:11.389972 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.389754    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z" event={"ID":"485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5","Type":"ContainerDied","Data":"d00e3f57549a92f3376469df7d6c9cb94159c1cf19037b838f89fe63f0ddaf4e"}
Oct 31 21:37:11.389972 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.389779    4460 scope.go:122] "RemoveContainer" containerID="9a712bedde325120c9746e740297a44c2df1db8e77e85dc6e2a463e8736a2782"
Oct 31 21:37:11.400382 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.400363    4460 scope.go:122] "RemoveContainer" containerID="10cbadfd2d786bbb06634b3e2368ad837f3385778fe526b74cc998e38dcfc6c4"
Oct 31 21:37:11.409542 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.409525    4460 scope.go:122] "RemoveContainer" containerID="9a712bedde325120c9746e740297a44c2df1db8e77e85dc6e2a463e8736a2782"
Oct 31 21:37:11.409827 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:37:11.409801    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a712bedde325120c9746e740297a44c2df1db8e77e85dc6e2a463e8736a2782\": container with ID starting with 9a712bedde325120c9746e740297a44c2df1db8e77e85dc6e2a463e8736a2782 not found: ID does not exist" containerID="9a712bedde325120c9746e740297a44c2df1db8e77e85dc6e2a463e8736a2782"
Oct 31 21:37:11.409874 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.409836    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a712bedde325120c9746e740297a44c2df1db8e77e85dc6e2a463e8736a2782"} err="failed to get container status \"9a712bedde325120c9746e740297a44c2df1db8e77e85dc6e2a463e8736a2782\": rpc error: code = NotFound desc = could not find container \"9a712bedde325120c9746e740297a44c2df1db8e77e85dc6e2a463e8736a2782\": container with ID starting with 9a712bedde325120c9746e740297a44c2df1db8e77e85dc6e2a463e8736a2782 not found: ID does not exist"
Oct 31 21:37:11.409874 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.409856    4460 scope.go:122] "RemoveContainer" containerID="10cbadfd2d786bbb06634b3e2368ad837f3385778fe526b74cc998e38dcfc6c4"
Oct 31 21:37:11.410137 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:37:11.410117    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10cbadfd2d786bbb06634b3e2368ad837f3385778fe526b74cc998e38dcfc6c4\": container with ID starting with 10cbadfd2d786bbb06634b3e2368ad837f3385778fe526b74cc998e38dcfc6c4 not found: ID does not exist" containerID="10cbadfd2d786bbb06634b3e2368ad837f3385778fe526b74cc998e38dcfc6c4"
Oct 31 21:37:11.410191 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.410144    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10cbadfd2d786bbb06634b3e2368ad837f3385778fe526b74cc998e38dcfc6c4"} err="failed to get container status \"10cbadfd2d786bbb06634b3e2368ad837f3385778fe526b74cc998e38dcfc6c4\": rpc error: code = NotFound desc = could not find container \"10cbadfd2d786bbb06634b3e2368ad837f3385778fe526b74cc998e38dcfc6c4\": container with ID starting with 10cbadfd2d786bbb06634b3e2368ad837f3385778fe526b74cc998e38dcfc6c4 not found: ID does not exist"
Oct 31 21:37:11.419569 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.419547    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:11.452962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.452929    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"]
Oct 31 21:37:11.524961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:11.514838    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-08ddf0-openshift-default-6ddc79869c-m4w7z"]
Oct 31 21:37:12.226184 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:12.226156    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5" path="/var/lib/kubelet/pods/485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5/volumes"
Oct 31 21:37:13.033560 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:13.033522    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:14.710715 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:14.710669    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:15.060407 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:15.060373    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:16.277498 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:16.277462    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-43fb1b-5799d848d7-m7b45"]
Oct 31 21:37:16.388570 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:16.388516    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:16.542680 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:16.542584    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:18.401016 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:18.400980    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:20.427964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:20.426108    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:20.954876 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:20.954839    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:21.280120 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:21.277752    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-43fb1b-5799d848d7-m7b45" podUID="850eb5d3-4057-4faa-a3ed-4b7e95f31108" containerName="payload-pre-processing" containerID="cri-o://05ef485fd64dc230102d0cd36ad4acf2d5412b8969b3c88e829a049fb39da3ca" gracePeriod=25
Oct 31 21:37:21.412984 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:21.412920    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:21.472031 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:37:21.471979    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:37:21.507506 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:21.507475    4460 generic.go:356] "Generic (PLEG): container finished" podID="850eb5d3-4057-4faa-a3ed-4b7e95f31108" containerID="05ef485fd64dc230102d0cd36ad4acf2d5412b8969b3c88e829a049fb39da3ca" exitCode=0
Oct 31 21:37:21.507682 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:21.507545    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-43fb1b-5799d848d7-m7b45" event={"ID":"850eb5d3-4057-4faa-a3ed-4b7e95f31108","Type":"ContainerDied","Data":"05ef485fd64dc230102d0cd36ad4acf2d5412b8969b3c88e829a049fb39da3ca"}
Oct 31 21:37:21.669843 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:21.669817    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-43fb1b-5799d848d7-m7b45"
Oct 31 21:37:21.786989 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:21.786925    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/850eb5d3-4057-4faa-a3ed-4b7e95f31108-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/850eb5d3-4057-4faa-a3ed-4b7e95f31108-plugins-config-volume\") pod \"850eb5d3-4057-4faa-a3ed-4b7e95f31108\" (UID: \"850eb5d3-4057-4faa-a3ed-4b7e95f31108\") "
Oct 31 21:37:21.787177 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:21.787160    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/850eb5d3-4057-4faa-a3ed-4b7e95f31108-kube-api-access-nzw85\" (UniqueName: \"kubernetes.io/projected/850eb5d3-4057-4faa-a3ed-4b7e95f31108-kube-api-access-nzw85\") pod \"850eb5d3-4057-4faa-a3ed-4b7e95f31108\" (UID: \"850eb5d3-4057-4faa-a3ed-4b7e95f31108\") "
Oct 31 21:37:21.787270 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:21.787245    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/850eb5d3-4057-4faa-a3ed-4b7e95f31108-plugins-config-volume" pod "850eb5d3-4057-4faa-a3ed-4b7e95f31108" (UID: "850eb5d3-4057-4faa-a3ed-4b7e95f31108"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:37:21.787551 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:21.787538    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/850eb5d3-4057-4faa-a3ed-4b7e95f31108-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:37:21.789398 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:21.789365    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/850eb5d3-4057-4faa-a3ed-4b7e95f31108-kube-api-access-nzw85" pod "850eb5d3-4057-4faa-a3ed-4b7e95f31108" (UID: "850eb5d3-4057-4faa-a3ed-4b7e95f31108"). InnerVolumeSpecName "kube-api-access-nzw85". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:37:21.859624 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:21.859586    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"]
Oct 31 21:37:21.860684 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:21.860656    4460 cpu_manager.go:414] "containerMap: removing container" podUID="485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5" containerName="istio-proxy"
Oct 31 21:37:21.860684 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:21.860683    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5" containerName="istio-proxy"
Oct 31 21:37:21.860881 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:21.860718    4460 cpu_manager.go:414] "containerMap: removing container" podUID="850eb5d3-4057-4faa-a3ed-4b7e95f31108" containerName="payload-pre-processing"
Oct 31 21:37:21.860881 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:21.860727    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="850eb5d3-4057-4faa-a3ed-4b7e95f31108" containerName="payload-pre-processing"
Oct 31 21:37:21.860881 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:21.860749    4460 cpu_manager.go:414] "containerMap: removing container" podUID="485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5" containerName="istio-proxy"
Oct 31 21:37:21.860881 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:21.860759    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5" containerName="istio-proxy"
Oct 31 21:37:21.861238 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:21.860903    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5" containerName="istio-proxy"
Oct 31 21:37:21.861238 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:21.860920    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="850eb5d3-4057-4faa-a3ed-4b7e95f31108" containerName="payload-pre-processing"
Oct 31 21:37:21.861238 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:21.860930    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="485bbdc1-0d53-47b1-9b33-1b89cbf7f5c5" containerName="istio-proxy"
Oct 31 21:37:21.864615 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:21.864595    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:37:21.888569 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:21.883336    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w4-c744c0-openshift-default-dockercfg-wmtrf\""
Oct 31 21:37:21.888569 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:21.887605    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"]
Oct 31 21:37:21.888569 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:21.888545    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzw85\" (UniqueName: \"kubernetes.io/projected/850eb5d3-4057-4faa-a3ed-4b7e95f31108-kube-api-access-nzw85\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:37:21.914924 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:21.914890    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:22.017964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:21.994038    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-credential-socket\") pod \"e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") " pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:37:22.018316 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.018285    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-istio-envoy\") pod \"e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") " pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:37:22.018545 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.018517    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae15a792-70d4-40c4-8222-5a81b6e885be-istio-podinfo\") pod \"e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") " pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:37:22.018743 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.018725    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7mt6\" (UniqueName: \"kubernetes.io/projected/ae15a792-70d4-40c4-8222-5a81b6e885be-kube-api-access-g7mt6\") pod \"e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") " pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:37:22.019076 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.019055    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-workload-socket\") pod \"e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") " pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:37:22.034022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.027213    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae15a792-70d4-40c4-8222-5a81b6e885be-istiod-ca-cert\") pod \"e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") " pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:37:22.034022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.027291    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-istio-data\") pod \"e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") " pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:37:22.034022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.027473    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae15a792-70d4-40c4-8222-5a81b6e885be-istio-token\") pod \"e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") " pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:37:22.034022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.027515    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-workload-certs\") pod \"e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") " pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:37:22.131269 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.131180    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae15a792-70d4-40c4-8222-5a81b6e885be-istio-token\") pod \"e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") " pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:37:22.132644 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.132615    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-workload-certs\") pod \"e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") " pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:37:22.132963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.132929    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-credential-socket\") pod \"e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") " pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:37:22.133097 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.133083    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-istio-envoy\") pod \"e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") " pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:37:22.133226 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.133212    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae15a792-70d4-40c4-8222-5a81b6e885be-istio-podinfo\") pod \"e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") " pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:37:22.133359 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.133346    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7mt6\" (UniqueName: \"kubernetes.io/projected/ae15a792-70d4-40c4-8222-5a81b6e885be-kube-api-access-g7mt6\") pod \"e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") " pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:37:22.133534 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.133517    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-workload-socket\") pod \"e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") " pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:37:22.139787 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.134643    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-workload-certs\") pod \"e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") " pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:37:22.139787 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.134862    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-credential-socket\") pod \"e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") " pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:37:22.139787 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.135805    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-workload-socket\") pod \"e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") " pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:37:22.144269 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.144247    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae15a792-70d4-40c4-8222-5a81b6e885be-istiod-ca-cert\") pod \"e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") " pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:37:22.149695 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.148246    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae15a792-70d4-40c4-8222-5a81b6e885be-istiod-ca-cert\") pod \"e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") " pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:37:22.165965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.150139    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-istio-data\") pod \"e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") " pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:37:22.165965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.152607    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae15a792-70d4-40c4-8222-5a81b6e885be-istio-podinfo\") pod \"e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") " pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:37:22.166756 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.149616    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-istio-data\") pod \"e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") " pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:37:22.166908 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.166882    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-istio-envoy\") pod \"e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") " pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:37:22.195051 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.194592    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:22.197521 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.195817    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7mt6\" (UniqueName: \"kubernetes.io/projected/ae15a792-70d4-40c4-8222-5a81b6e885be-kube-api-access-g7mt6\") pod \"e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") " pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:37:22.206978 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.203686    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae15a792-70d4-40c4-8222-5a81b6e885be-istio-token\") pod \"e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") " pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:37:22.461841 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.461749    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:22.479091 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.479064    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:37:22.599521 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.599474    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-43fb1b-5799d848d7-m7b45" event={"ID":"850eb5d3-4057-4faa-a3ed-4b7e95f31108","Type":"ContainerDied","Data":"828dab523690524d0597f32df22c04f3704677999fbc16dec0a448c4c7b52158"}
Oct 31 21:37:22.599521 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.599549    4460 scope.go:122] "RemoveContainer" containerID="05ef485fd64dc230102d0cd36ad4acf2d5412b8969b3c88e829a049fb39da3ca"
Oct 31 21:37:22.599829 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.599736    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-43fb1b-5799d848d7-m7b45"
Oct 31 21:37:22.693089 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.693044    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-43fb1b-5799d848d7-m7b45"]
Oct 31 21:37:22.696664 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.696633    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-43fb1b-5799d848d7-m7b45"]
Oct 31 21:37:22.979501 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.979471    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"]
Oct 31 21:37:22.979910 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:37:22.979883    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae15a792_70d4_40c4_8222_5a81b6e885be.slice/crio-003ab786806022397c85ebdb77d7a3e6cfccace56f76b27bc31193c7c2fa6a93 WatchSource:0}: Error finding container 003ab786806022397c85ebdb77d7a3e6cfccace56f76b27bc31193c7c2fa6a93: Status 404 returned error can't find the container with id 003ab786806022397c85ebdb77d7a3e6cfccace56f76b27bc31193c7c2fa6a93
Oct 31 21:37:22.982464 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.982437    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:37:22.982538 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.982495    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:37:22.982538 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:22.982531    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:37:23.607796 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:23.607754    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6" event={"ID":"ae15a792-70d4-40c4-8222-5a81b6e885be","Type":"ContainerStarted","Data":"e5346db7c8cd8b6dc928e9fc6064bcce5d46b3cd488c92e68e7104ecba4b5107"}
Oct 31 21:37:23.608324 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:23.607840    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6" event={"ID":"ae15a792-70d4-40c4-8222-5a81b6e885be","Type":"ContainerStarted","Data":"003ab786806022397c85ebdb77d7a3e6cfccace56f76b27bc31193c7c2fa6a93"}
Oct 31 21:37:23.709601 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:23.709552    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6" podStartSLOduration=2.7095369910000002 podStartE2EDuration="2.709536991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:37:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:37:23.693157722 +0000 UTC m=+1283.609467149" watchObservedRunningTime="2026-10-31 21:37:23.709536991 +0000 UTC m=+1283.625846418"
Oct 31 21:37:24.239494 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:24.239460    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:24.248640 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:24.248602    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="850eb5d3-4057-4faa-a3ed-4b7e95f31108" path="/var/lib/kubelet/pods/850eb5d3-4057-4faa-a3ed-4b7e95f31108/volumes"
Oct 31 21:37:24.479648 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:24.479616    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:37:24.486012 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:24.485990    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:37:24.615406 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:24.615374    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:37:24.616202 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:24.616183    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:37:26.060921 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:26.060884    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:26.172737 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:26.172693    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:27.803723 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:27.803674    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:27.914245 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:27.914198    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:29.521336 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:29.521300    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:30.034071 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:30.034035    4460 patch_prober.go:29] interesting pod/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 21:37:30.034248 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:30.034091    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm" podUID="9a09d750-0cc0-430b-a2d6-6490cf9f1361" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:37:30.709497 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:30.709417    4460 generic.go:356] "Generic (PLEG): container finished" podID="9a09d750-0cc0-430b-a2d6-6490cf9f1361" containerID="3042a385bef9cf71215d9cac8aa2098c635c2f82effbb7d7b18e08bb1726d6ec" exitCode=0
Oct 31 21:37:30.709894 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:30.709493    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm" event={"ID":"9a09d750-0cc0-430b-a2d6-6490cf9f1361","Type":"ContainerDied","Data":"3042a385bef9cf71215d9cac8aa2098c635c2f82effbb7d7b18e08bb1726d6ec"}
Oct 31 21:37:30.710003 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:30.709986    4460 scope.go:122] "RemoveContainer" containerID="3042a385bef9cf71215d9cac8aa2098c635c2f82effbb7d7b18e08bb1726d6ec"
Oct 31 21:37:30.711535 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:30.711501    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:37:30.711641 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:30.711591    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:37:30.711704 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:30.711638    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:37:31.209394 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:31.209358    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:31.719043 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:31.719006    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm" event={"ID":"9a09d750-0cc0-430b-a2d6-6490cf9f1361","Type":"ContainerStarted","Data":"9d2ccabc9072b0943ccae28d396621786180a78ffdc1dfec174c660f97615d8f"}
Oct 31 21:37:31.846171 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:31.846134    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:32.301319 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:32.301279    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:32.369964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:32.363454    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:37:32.369964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:32.363599    4460 patch_prober.go:29] interesting pod/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 31 21:37:32.369964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:32.363654    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm" podUID="9a09d750-0cc0-430b-a2d6-6490cf9f1361" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 31 21:37:32.756025 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:32.753918    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:32.942498 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:32.942426    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:33.359465 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:33.359433    4460 patch_prober.go:29] interesting pod/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 31 21:37:33.359647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:33.359497    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm" podUID="9a09d750-0cc0-430b-a2d6-6490cf9f1361" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 31 21:37:34.359929 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:34.359893    4460 patch_prober.go:29] interesting pod/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 31 21:37:34.360450 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:34.359980    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm" podUID="9a09d750-0cc0-430b-a2d6-6490cf9f1361" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 31 21:37:34.609168 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:34.609131    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:35.359558 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:35.359521    4460 patch_prober.go:29] interesting pod/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 31 21:37:35.359740 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:35.359595    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm" podUID="9a09d750-0cc0-430b-a2d6-6490cf9f1361" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 31 21:37:36.359621 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:36.359588    4460 patch_prober.go:29] interesting pod/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Oct 31 21:37:36.360042 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:36.359646    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm" podUID="9a09d750-0cc0-430b-a2d6-6490cf9f1361" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Oct 31 21:37:36.410180 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:36.410144    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:36.541008 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:36.538893    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:36.650933 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:36.650851    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:37.363826 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:37.363795    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:37:37.364319 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:37.364186    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:37:37.364804 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:37.364784    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 21:37:38.051887 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:38.051852    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf"]
Oct 31 21:37:38.056431 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:38.056409    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf"
Oct 31 21:37:38.094844 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:38.082669    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w4-c744c0-serving-cert\""
Oct 31 21:37:38.094844 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:38.088754    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf"]
Oct 31 21:37:38.130335 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:38.130303    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9091a526-1fb8-494b-a486-6c4e67dd9953-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf\" (UID: \"9091a526-1fb8-494b-a486-6c4e67dd9953\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf"
Oct 31 21:37:38.130551 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:38.130523    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9091a526-1fb8-494b-a486-6c4e67dd9953-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf\" (UID: \"9091a526-1fb8-494b-a486-6c4e67dd9953\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf"
Oct 31 21:37:38.130638 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:38.130621    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9091a526-1fb8-494b-a486-6c4e67dd9953-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf\" (UID: \"9091a526-1fb8-494b-a486-6c4e67dd9953\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf"
Oct 31 21:37:38.130738 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:38.130723    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2r6lz\" (UniqueName: \"kubernetes.io/projected/9091a526-1fb8-494b-a486-6c4e67dd9953-kube-api-access-2r6lz\") pod \"maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf\" (UID: \"9091a526-1fb8-494b-a486-6c4e67dd9953\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf"
Oct 31 21:37:38.145396 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:38.145305    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:38.232466 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:38.232431    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9091a526-1fb8-494b-a486-6c4e67dd9953-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf\" (UID: \"9091a526-1fb8-494b-a486-6c4e67dd9953\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf"
Oct 31 21:37:38.232639 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:38.232527    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r6lz\" (UniqueName: \"kubernetes.io/projected/9091a526-1fb8-494b-a486-6c4e67dd9953-kube-api-access-2r6lz\") pod \"maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf\" (UID: \"9091a526-1fb8-494b-a486-6c4e67dd9953\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf"
Oct 31 21:37:38.232709 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:38.232673    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9091a526-1fb8-494b-a486-6c4e67dd9953-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf\" (UID: \"9091a526-1fb8-494b-a486-6c4e67dd9953\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf"
Oct 31 21:37:38.232811 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:38.232797    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9091a526-1fb8-494b-a486-6c4e67dd9953-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf\" (UID: \"9091a526-1fb8-494b-a486-6c4e67dd9953\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf"
Oct 31 21:37:38.233110 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:38.233083    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9091a526-1fb8-494b-a486-6c4e67dd9953-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf\" (UID: \"9091a526-1fb8-494b-a486-6c4e67dd9953\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf"
Oct 31 21:37:38.234934 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:38.234912    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9091a526-1fb8-494b-a486-6c4e67dd9953-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf\" (UID: \"9091a526-1fb8-494b-a486-6c4e67dd9953\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf"
Oct 31 21:37:38.235121 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:38.235100    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9091a526-1fb8-494b-a486-6c4e67dd9953-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf\" (UID: \"9091a526-1fb8-494b-a486-6c4e67dd9953\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf"
Oct 31 21:37:38.258739 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:38.258718    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:38.261022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:38.261001    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2r6lz\" (UniqueName: \"kubernetes.io/projected/9091a526-1fb8-494b-a486-6c4e67dd9953-kube-api-access-2r6lz\") pod \"maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf\" (UID: \"9091a526-1fb8-494b-a486-6c4e67dd9953\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf"
Oct 31 21:37:38.370852 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:38.370784    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf"
Oct 31 21:37:38.986088 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:38.986061    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf"]
Oct 31 21:37:38.987766 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:37:38.987733    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9091a526_1fb8_494b_a486_6c4e67dd9953.slice/crio-a90275d5ca83f758571ea7277d80b5847e986650c74b1932fc24b8e13142afd4 WatchSource:0}: Error finding container a90275d5ca83f758571ea7277d80b5847e986650c74b1932fc24b8e13142afd4: Status 404 returned error can't find the container with id a90275d5ca83f758571ea7277d80b5847e986650c74b1932fc24b8e13142afd4
Oct 31 21:37:39.566463 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:39.566379    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-43fb1b-946d9c597-ddppm"]
Oct 31 21:37:39.567269 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:39.567236    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-43fb1b-946d9c597-ddppm" podUID="c800a8c6-2786-4aaf-ade9-07a46af9cc94" containerName="maas-api" containerID="cri-o://9574a25ecd4f5cdee33275ed86cf26ac835f3fe1e2bf9bfa75ebcb172fca9e3a" gracePeriod=30
Oct 31 21:37:39.616562 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:39.616532    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:39.929322 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:39.929180    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf" event={"ID":"9091a526-1fb8-494b-a486-6c4e67dd9953","Type":"ContainerStarted","Data":"f2dbac6c121eafdb29baf0ee04d09221c8e8303dd02ce7548e5815c336c94d04"}
Oct 31 21:37:39.929322 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:39.929266    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf" event={"ID":"9091a526-1fb8-494b-a486-6c4e67dd9953","Type":"ContainerStarted","Data":"a90275d5ca83f758571ea7277d80b5847e986650c74b1932fc24b8e13142afd4"}
Oct 31 21:37:39.929548 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:39.929338    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf"
Oct 31 21:37:39.950097 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:39.949864    4460 generic.go:356] "Generic (PLEG): container finished" podID="c800a8c6-2786-4aaf-ade9-07a46af9cc94" containerID="9574a25ecd4f5cdee33275ed86cf26ac835f3fe1e2bf9bfa75ebcb172fca9e3a" exitCode=0
Oct 31 21:37:39.950097 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:39.950028    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-43fb1b-946d9c597-ddppm" event={"ID":"c800a8c6-2786-4aaf-ade9-07a46af9cc94","Type":"ContainerDied","Data":"9574a25ecd4f5cdee33275ed86cf26ac835f3fe1e2bf9bfa75ebcb172fca9e3a"}
Oct 31 21:37:40.037029 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:40.034023    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf" podStartSLOduration=1.70630565 podStartE2EDuration="2.034002503s" totalImagesPullingTime="327.696853ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:37:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:37:39.992338211 +0000 UTC m=+1299.908647640" watchObservedRunningTime="2026-10-31 21:37:40.034002503 +0000 UTC m=+1299.950311933"
Oct 31 21:37:40.048466 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:40.047663    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-43fb1b-946d9c597-ddppm"
Oct 31 21:37:40.090529 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:40.090493    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:40.201902 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:40.201819    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:40.215232 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:40.215203    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c800a8c6-2786-4aaf-ade9-07a46af9cc94-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c800a8c6-2786-4aaf-ade9-07a46af9cc94-maas-api-tls\") pod \"c800a8c6-2786-4aaf-ade9-07a46af9cc94\" (UID: \"c800a8c6-2786-4aaf-ade9-07a46af9cc94\") "
Oct 31 21:37:40.215370 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:40.215316    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c800a8c6-2786-4aaf-ade9-07a46af9cc94-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c800a8c6-2786-4aaf-ade9-07a46af9cc94-metrics-certs\") pod \"c800a8c6-2786-4aaf-ade9-07a46af9cc94\" (UID: \"c800a8c6-2786-4aaf-ade9-07a46af9cc94\") "
Oct 31 21:37:40.215495 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:40.215481    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c800a8c6-2786-4aaf-ade9-07a46af9cc94-kube-api-access-pgw25\" (UniqueName: \"kubernetes.io/projected/c800a8c6-2786-4aaf-ade9-07a46af9cc94-kube-api-access-pgw25\") pod \"c800a8c6-2786-4aaf-ade9-07a46af9cc94\" (UID: \"c800a8c6-2786-4aaf-ade9-07a46af9cc94\") "
Oct 31 21:37:40.215586 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:40.215571    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c800a8c6-2786-4aaf-ade9-07a46af9cc94-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c800a8c6-2786-4aaf-ade9-07a46af9cc94-openshift-service-ca\") pod \"c800a8c6-2786-4aaf-ade9-07a46af9cc94\" (UID: \"c800a8c6-2786-4aaf-ade9-07a46af9cc94\") "
Oct 31 21:37:40.216071 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:40.216038    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c800a8c6-2786-4aaf-ade9-07a46af9cc94-openshift-service-ca" pod "c800a8c6-2786-4aaf-ade9-07a46af9cc94" (UID: "c800a8c6-2786-4aaf-ade9-07a46af9cc94"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:37:40.216315 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:40.216298    4460 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c800a8c6-2786-4aaf-ade9-07a46af9cc94-openshift-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:37:40.217666 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:40.217638    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c800a8c6-2786-4aaf-ade9-07a46af9cc94-maas-api-tls" pod "c800a8c6-2786-4aaf-ade9-07a46af9cc94" (UID: "c800a8c6-2786-4aaf-ade9-07a46af9cc94"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 21:37:40.217884 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:40.217860    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c800a8c6-2786-4aaf-ade9-07a46af9cc94-kube-api-access-pgw25" pod "c800a8c6-2786-4aaf-ade9-07a46af9cc94" (UID: "c800a8c6-2786-4aaf-ade9-07a46af9cc94"). InnerVolumeSpecName "kube-api-access-pgw25". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:37:40.218037 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:40.217899    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c800a8c6-2786-4aaf-ade9-07a46af9cc94-metrics-certs" pod "c800a8c6-2786-4aaf-ade9-07a46af9cc94" (UID: "c800a8c6-2786-4aaf-ade9-07a46af9cc94"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 21:37:40.321961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:40.319589    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgw25\" (UniqueName: \"kubernetes.io/projected/c800a8c6-2786-4aaf-ade9-07a46af9cc94-kube-api-access-pgw25\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:37:40.321961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:40.319631    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c800a8c6-2786-4aaf-ade9-07a46af9cc94-maas-api-tls\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:37:40.321961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:40.319646    4460 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c800a8c6-2786-4aaf-ade9-07a46af9cc94-metrics-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:37:40.351961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:40.349859    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:40.707847 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:40.707814    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:40.958313 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:40.958234    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-43fb1b-946d9c597-ddppm"
Oct 31 21:37:40.958313 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:40.958258    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-43fb1b-946d9c597-ddppm" event={"ID":"c800a8c6-2786-4aaf-ade9-07a46af9cc94","Type":"ContainerDied","Data":"7e0e95b763a9caa2a12af7b79191978eb891b629fcd9062f0fafa90354a91b8a"}
Oct 31 21:37:40.958544 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:40.958344    4460 scope.go:122] "RemoveContainer" containerID="9574a25ecd4f5cdee33275ed86cf26ac835f3fe1e2bf9bfa75ebcb172fca9e3a"
Oct 31 21:37:41.016406 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:41.016379    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-43fb1b-946d9c597-ddppm"]
Oct 31 21:37:41.029870 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:41.029844    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-43fb1b-946d9c597-ddppm"]
Oct 31 21:37:41.968370 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:41.968338    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:42.225876 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:42.225796    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c800a8c6-2786-4aaf-ade9-07a46af9cc94" path="/var/lib/kubelet/pods/c800a8c6-2786-4aaf-ade9-07a46af9cc94/volumes"
Oct 31 21:37:42.791670 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:42.791628    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:37:43.775507 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:43.775468    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:45.473542 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:45.473512    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:45.965819 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:45.965786    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf"
Oct 31 21:37:46.091570 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:46.091535    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:46.212419 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:46.212381    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:47.216648 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:47.216611    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:47.364248 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:47.355984    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:47.467955 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:47.467855    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:47.662140 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:47.662077    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"]
Oct 31 21:37:47.662386 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:47.662350    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs" podUID="dd82789d-81ca-487e-8bfb-e2567ca866bd" containerName="istio-proxy" containerID="cri-o://0ad93cdd8a1238afcb38b58a313cfc4496716b90a56093f52b7d6cb54a832cf5" gracePeriod=30
Oct 31 21:37:47.702480 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:47.702450    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:48.673881 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:48.673839    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:48.957524 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:48.957435    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:49.096718 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.096680    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"]
Oct 31 21:37:49.097417 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.097394    4460 cpu_manager.go:414] "containerMap: removing container" podUID="c800a8c6-2786-4aaf-ade9-07a46af9cc94" containerName="maas-api"
Oct 31 21:37:49.097417 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.097413    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c800a8c6-2786-4aaf-ade9-07a46af9cc94" containerName="maas-api"
Oct 31 21:37:49.097600 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.097519    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="c800a8c6-2786-4aaf-ade9-07a46af9cc94" containerName="maas-api"
Oct 31 21:37:49.103001 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.102971    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:37:49.125552 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.125518    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"]
Oct 31 21:37:49.134404 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.134372    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-89d4a2-openshift-default-dockercfg-fbz8w\""
Oct 31 21:37:49.146901 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.146872    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:49.264024 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.260887    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istio-podinfo\") pod \"e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") " pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:37:49.264024 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.261006    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d85g2\" (UniqueName: \"kubernetes.io/projected/b49ec684-cafd-4063-9ffa-c3884fbe7abd-kube-api-access-d85g2\") pod \"e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") " pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:37:49.264024 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.261047    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istiod-ca-cert\") pod \"e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") " pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:37:49.264024 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.261122    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-credential-socket\") pod \"e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") " pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:37:49.264024 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.261193    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-workload-certs\") pod \"e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") " pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:37:49.264024 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.261224    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istio-envoy\") pod \"e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") " pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:37:49.264024 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.261434    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istio-token\") pod \"e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") " pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:37:49.264024 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.261479    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-workload-socket\") pod \"e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") " pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:37:49.264024 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.261507    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istio-data\") pod \"e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") " pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:37:49.290249 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.290198    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:49.369964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.369372    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istio-token\") pod \"e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") " pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:37:49.369964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.369438    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-workload-socket\") pod \"e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") " pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:37:49.369964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.369466    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istio-data\") pod \"e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") " pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:37:49.369964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.369590    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istio-podinfo\") pod \"e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") " pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:37:49.369964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.369636    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d85g2\" (UniqueName: \"kubernetes.io/projected/b49ec684-cafd-4063-9ffa-c3884fbe7abd-kube-api-access-d85g2\") pod \"e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") " pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:37:49.369964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.369663    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istiod-ca-cert\") pod \"e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") " pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:37:49.369964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.369748    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-credential-socket\") pod \"e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") " pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:37:49.369964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.369837    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-workload-certs\") pod \"e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") " pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:37:49.369964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.369869    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istio-envoy\") pod \"e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") " pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:37:49.373570 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.373539    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istio-data\") pod \"e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") " pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:37:49.374043 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.374018    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-credential-socket\") pod \"e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") " pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:37:49.389610 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.389576    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istio-podinfo\") pod \"e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") " pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:37:49.397360 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.390538    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-workload-certs\") pod \"e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") " pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:37:49.397360 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.391359    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istiod-ca-cert\") pod \"e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") " pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:37:49.397360 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.391771    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-workload-socket\") pod \"e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") " pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:37:49.401745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.401716    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d85g2\" (UniqueName: \"kubernetes.io/projected/b49ec684-cafd-4063-9ffa-c3884fbe7abd-kube-api-access-d85g2\") pod \"e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") " pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:37:49.402596 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.402515    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istio-token\") pod \"e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") " pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:37:49.429401 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.429331    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istio-envoy\") pod \"e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") " pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:37:49.489817 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.489781    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:49.731383 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:49.731283    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:37:50.009312 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:50.005870    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:50.202049 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:50.202011    4460 patch_prober.go:29] interesting pod/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 21:37:50.203913 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:50.203883    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs" podUID="dd82789d-81ca-487e-8bfb-e2567ca866bd" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 21:37:50.252683 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:50.252650    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:50.299231 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:50.299204    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"]
Oct 31 21:37:50.299763 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:37:50.299719    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb49ec684_cafd_4063_9ffa_c3884fbe7abd.slice/crio-6d690c8ece2811177d0eb0e1deb4a89b4dd0ff1c02476127912e1beb0cfb5b60 WatchSource:0}: Error finding container 6d690c8ece2811177d0eb0e1deb4a89b4dd0ff1c02476127912e1beb0cfb5b60: Status 404 returned error can't find the container with id 6d690c8ece2811177d0eb0e1deb4a89b4dd0ff1c02476127912e1beb0cfb5b60
Oct 31 21:37:50.302241 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:50.302203    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:37:50.302402 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:50.302285    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:37:50.302402 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:50.302330    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:37:50.688216 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:50.688126    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:50.833961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:50.829465    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:51.039517 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:51.039480    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:51.056156 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:51.056118    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd" event={"ID":"b49ec684-cafd-4063-9ffa-c3884fbe7abd","Type":"ContainerStarted","Data":"c1073b04fdbe5c21ec9137e8bf45d09178688b7382975481c53333333a7d29b1"}
Oct 31 21:37:51.056414 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:51.056183    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd" event={"ID":"b49ec684-cafd-4063-9ffa-c3884fbe7abd","Type":"ContainerStarted","Data":"6d690c8ece2811177d0eb0e1deb4a89b4dd0ff1c02476127912e1beb0cfb5b60"}
Oct 31 21:37:51.108408 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:51.108349    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd" podStartSLOduration=2.108331821 podStartE2EDuration="2.108331821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:37:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:37:51.103165855 +0000 UTC m=+1311.019475288" watchObservedRunningTime="2026-10-31 21:37:51.108331821 +0000 UTC m=+1311.024641250"
Oct 31 21:37:51.588862 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:37:51.587332    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:37:51.734207 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:51.734170    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:37:51.740467 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:51.740441    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:37:52.063025 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:52.062992    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:37:52.064058 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:52.064032    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:37:52.694590 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:52.694536    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:53.035506 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.035472    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:37:53.049772 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.049747    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-workload-certs\") pod \"dd82789d-81ca-487e-8bfb-e2567ca866bd\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") "
Oct 31 21:37:53.049878 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.049784    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-workload-socket\") pod \"dd82789d-81ca-487e-8bfb-e2567ca866bd\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") "
Oct 31 21:37:53.049878 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.049861    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dd82789d-81ca-487e-8bfb-e2567ca866bd-istio-token\" (UniqueName: \"kubernetes.io/projected/dd82789d-81ca-487e-8bfb-e2567ca866bd-istio-token\") pod \"dd82789d-81ca-487e-8bfb-e2567ca866bd\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") "
Oct 31 21:37:53.049995 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.049888    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/dd82789d-81ca-487e-8bfb-e2567ca866bd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd82789d-81ca-487e-8bfb-e2567ca866bd-istio-podinfo\") pod \"dd82789d-81ca-487e-8bfb-e2567ca866bd\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") "
Oct 31 21:37:53.049995 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.049925    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-istio-data\") pod \"dd82789d-81ca-487e-8bfb-e2567ca866bd\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") "
Oct 31 21:37:53.049995 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.049976    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dd82789d-81ca-487e-8bfb-e2567ca866bd-kube-api-access-7f976\" (UniqueName: \"kubernetes.io/projected/dd82789d-81ca-487e-8bfb-e2567ca866bd-kube-api-access-7f976\") pod \"dd82789d-81ca-487e-8bfb-e2567ca866bd\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") "
Oct 31 21:37:53.050149 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.050012    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-workload-certs" pod "dd82789d-81ca-487e-8bfb-e2567ca866bd" (UID: "dd82789d-81ca-487e-8bfb-e2567ca866bd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:37:53.050149 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.050086    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dd82789d-81ca-487e-8bfb-e2567ca866bd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd82789d-81ca-487e-8bfb-e2567ca866bd-istiod-ca-cert\") pod \"dd82789d-81ca-487e-8bfb-e2567ca866bd\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") "
Oct 31 21:37:53.050257 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.050149    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-credential-socket\") pod \"dd82789d-81ca-487e-8bfb-e2567ca866bd\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") "
Oct 31 21:37:53.050257 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.050171    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-workload-socket" pod "dd82789d-81ca-487e-8bfb-e2567ca866bd" (UID: "dd82789d-81ca-487e-8bfb-e2567ca866bd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:37:53.050366 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.050291    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-istio-envoy\") pod \"dd82789d-81ca-487e-8bfb-e2567ca866bd\" (UID: \"dd82789d-81ca-487e-8bfb-e2567ca866bd\") "
Oct 31 21:37:53.050366 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.050299    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-istio-data" pod "dd82789d-81ca-487e-8bfb-e2567ca866bd" (UID: "dd82789d-81ca-487e-8bfb-e2567ca866bd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:37:53.050467 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.050412    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-credential-socket" pod "dd82789d-81ca-487e-8bfb-e2567ca866bd" (UID: "dd82789d-81ca-487e-8bfb-e2567ca866bd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:37:53.050467 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.050453    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd82789d-81ca-487e-8bfb-e2567ca866bd-istiod-ca-cert" pod "dd82789d-81ca-487e-8bfb-e2567ca866bd" (UID: "dd82789d-81ca-487e-8bfb-e2567ca866bd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:37:53.050819 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.050799    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:37:53.050905 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.050824    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:37:53.050905 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.050837    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:37:53.050905 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.050845    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:37:53.050905 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.050853    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd82789d-81ca-487e-8bfb-e2567ca866bd-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:37:53.052249 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.052225    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dd82789d-81ca-487e-8bfb-e2567ca866bd-istio-podinfo" pod "dd82789d-81ca-487e-8bfb-e2567ca866bd" (UID: "dd82789d-81ca-487e-8bfb-e2567ca866bd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:37:53.052382 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.052361    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd82789d-81ca-487e-8bfb-e2567ca866bd-istio-token" pod "dd82789d-81ca-487e-8bfb-e2567ca866bd" (UID: "dd82789d-81ca-487e-8bfb-e2567ca866bd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:37:53.052590 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.052563    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd82789d-81ca-487e-8bfb-e2567ca866bd-kube-api-access-7f976" pod "dd82789d-81ca-487e-8bfb-e2567ca866bd" (UID: "dd82789d-81ca-487e-8bfb-e2567ca866bd"). InnerVolumeSpecName "kube-api-access-7f976". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:37:53.052711 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.052694    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-istio-envoy" pod "dd82789d-81ca-487e-8bfb-e2567ca866bd" (UID: "dd82789d-81ca-487e-8bfb-e2567ca866bd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:37:53.070713 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.070687    4460 generic.go:356] "Generic (PLEG): container finished" podID="dd82789d-81ca-487e-8bfb-e2567ca866bd" containerID="0ad93cdd8a1238afcb38b58a313cfc4496716b90a56093f52b7d6cb54a832cf5" exitCode=0
Oct 31 21:37:53.071061 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.070743    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"
Oct 31 21:37:53.071061 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.070765    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs" event={"ID":"dd82789d-81ca-487e-8bfb-e2567ca866bd","Type":"ContainerDied","Data":"0ad93cdd8a1238afcb38b58a313cfc4496716b90a56093f52b7d6cb54a832cf5"}
Oct 31 21:37:53.071061 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.070840    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs" event={"ID":"dd82789d-81ca-487e-8bfb-e2567ca866bd","Type":"ContainerDied","Data":"94f630e217c04b28264a43bb6368ae49af3c4331d81c417b8fb979abb75d0a6f"}
Oct 31 21:37:53.071061 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.070862    4460 scope.go:122] "RemoveContainer" containerID="0ad93cdd8a1238afcb38b58a313cfc4496716b90a56093f52b7d6cb54a832cf5"
Oct 31 21:37:53.081533 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.081514    4460 scope.go:122] "RemoveContainer" containerID="dc44e026b70ed551abcbbbc17389ec363ff129c85ceeb95bb992464be87eabdb"
Oct 31 21:37:53.090777 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.090758    4460 scope.go:122] "RemoveContainer" containerID="0ad93cdd8a1238afcb38b58a313cfc4496716b90a56093f52b7d6cb54a832cf5"
Oct 31 21:37:53.091099 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:37:53.091075    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ad93cdd8a1238afcb38b58a313cfc4496716b90a56093f52b7d6cb54a832cf5\": container with ID starting with 0ad93cdd8a1238afcb38b58a313cfc4496716b90a56093f52b7d6cb54a832cf5 not found: ID does not exist" containerID="0ad93cdd8a1238afcb38b58a313cfc4496716b90a56093f52b7d6cb54a832cf5"
Oct 31 21:37:53.091197 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.091105    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ad93cdd8a1238afcb38b58a313cfc4496716b90a56093f52b7d6cb54a832cf5"} err="failed to get container status \"0ad93cdd8a1238afcb38b58a313cfc4496716b90a56093f52b7d6cb54a832cf5\": rpc error: code = NotFound desc = could not find container \"0ad93cdd8a1238afcb38b58a313cfc4496716b90a56093f52b7d6cb54a832cf5\": container with ID starting with 0ad93cdd8a1238afcb38b58a313cfc4496716b90a56093f52b7d6cb54a832cf5 not found: ID does not exist"
Oct 31 21:37:53.091197 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.091124    4460 scope.go:122] "RemoveContainer" containerID="dc44e026b70ed551abcbbbc17389ec363ff129c85ceeb95bb992464be87eabdb"
Oct 31 21:37:53.091358 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:37:53.091337    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc44e026b70ed551abcbbbc17389ec363ff129c85ceeb95bb992464be87eabdb\": container with ID starting with dc44e026b70ed551abcbbbc17389ec363ff129c85ceeb95bb992464be87eabdb not found: ID does not exist" containerID="dc44e026b70ed551abcbbbc17389ec363ff129c85ceeb95bb992464be87eabdb"
Oct 31 21:37:53.091399 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.091363    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc44e026b70ed551abcbbbc17389ec363ff129c85ceeb95bb992464be87eabdb"} err="failed to get container status \"dc44e026b70ed551abcbbbc17389ec363ff129c85ceeb95bb992464be87eabdb\": rpc error: code = NotFound desc = could not find container \"dc44e026b70ed551abcbbbc17389ec363ff129c85ceeb95bb992464be87eabdb\": container with ID starting with dc44e026b70ed551abcbbbc17389ec363ff129c85ceeb95bb992464be87eabdb not found: ID does not exist"
Oct 31 21:37:53.154256 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.154221    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"]
Oct 31 21:37:53.155886 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.155843    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd82789d-81ca-487e-8bfb-e2567ca866bd-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:37:53.155886 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.155874    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd82789d-81ca-487e-8bfb-e2567ca866bd-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:37:53.155886 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.155888    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd82789d-81ca-487e-8bfb-e2567ca866bd-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:37:53.156214 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.155901    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7f976\" (UniqueName: \"kubernetes.io/projected/dd82789d-81ca-487e-8bfb-e2567ca866bd-kube-api-access-7f976\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:37:53.162679 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.162651    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-43fb1b-openshift-default-644c8cb85f-tzdfs"]
Oct 31 21:37:53.219076 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.217524    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:53.427969 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.422864    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:53.861785 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.861754    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:53.985397 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:53.985353    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:54.183008 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:54.182914    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:54.225767 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:54.225734    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dd82789d-81ca-487e-8bfb-e2567ca866bd" path="/var/lib/kubelet/pods/dd82789d-81ca-487e-8bfb-e2567ca866bd/volumes"
Oct 31 21:37:54.529684 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:54.529652    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:55.467598 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:55.467565    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:55.580718 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:55.580682    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:55.640617 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:55.640580    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c40fc3-584b94f575-mkpjp"]
Oct 31 21:37:55.641552 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:55.641528    4460 cpu_manager.go:414] "containerMap: removing container" podUID="dd82789d-81ca-487e-8bfb-e2567ca866bd" containerName="istio-proxy"
Oct 31 21:37:55.641552 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:55.641552    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dd82789d-81ca-487e-8bfb-e2567ca866bd" containerName="istio-proxy"
Oct 31 21:37:55.641746 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:55.641735    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="dd82789d-81ca-487e-8bfb-e2567ca866bd" containerName="istio-proxy"
Oct 31 21:37:55.641795 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:55.641751    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="dd82789d-81ca-487e-8bfb-e2567ca866bd" containerName="istio-proxy"
Oct 31 21:37:55.646984 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:55.646965    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c40fc3-584b94f575-mkpjp"
Oct 31 21:37:55.665386 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:55.665361    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c40fc3-584b94f575-mkpjp"]
Oct 31 21:37:55.675118 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:55.675088    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-c40fc3-dockercfg-d6glj\""
Oct 31 21:37:55.675249 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:55.675122    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-c40fc3\""
Oct 31 21:37:55.706013 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:55.705978    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-c40fc3-584b94f575-mkpjp\" (UID: \"387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c40fc3-584b94f575-mkpjp"
Oct 31 21:37:55.706220 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:55.706195    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxzck\" (UniqueName: \"kubernetes.io/projected/387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8-kube-api-access-dxzck\") pod \"payload-pre-processing-e2e-ipp-a-c40fc3-584b94f575-mkpjp\" (UID: \"387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c40fc3-584b94f575-mkpjp"
Oct 31 21:37:55.833999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:55.814857    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-c40fc3-584b94f575-mkpjp\" (UID: \"387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c40fc3-584b94f575-mkpjp"
Oct 31 21:37:55.833999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:55.815125    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxzck\" (UniqueName: \"kubernetes.io/projected/387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8-kube-api-access-dxzck\") pod \"payload-pre-processing-e2e-ipp-a-c40fc3-584b94f575-mkpjp\" (UID: \"387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c40fc3-584b94f575-mkpjp"
Oct 31 21:37:55.833999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:55.816417    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-c40fc3-584b94f575-mkpjp\" (UID: \"387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c40fc3-584b94f575-mkpjp"
Oct 31 21:37:55.895964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:55.889540    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxzck\" (UniqueName: \"kubernetes.io/projected/387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8-kube-api-access-dxzck\") pod \"payload-pre-processing-e2e-ipp-a-c40fc3-584b94f575-mkpjp\" (UID: \"387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c40fc3-584b94f575-mkpjp"
Oct 31 21:37:55.965723 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:55.962052    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c40fc3-5fb8bfbcff-tt9qf"]
Oct 31 21:37:55.965723 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:55.963871    4460 cpu_manager.go:414] "containerMap: removing container" podUID="dd82789d-81ca-487e-8bfb-e2567ca866bd" containerName="istio-proxy"
Oct 31 21:37:55.965723 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:55.963903    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dd82789d-81ca-487e-8bfb-e2567ca866bd" containerName="istio-proxy"
Oct 31 21:37:56.025963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:56.015038    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-c40fc3-5fb8bfbcff-tt9qf"
Oct 31 21:37:56.053689 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:56.053652    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c40fc3-5fb8bfbcff-tt9qf"]
Oct 31 21:37:56.058982 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:56.058613    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c40fc3-584b94f575-mkpjp"
Oct 31 21:37:56.125913 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:56.125875    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8qwj\" (UniqueName: \"kubernetes.io/projected/bf4909ee-383b-4d58-9232-127bb722e696-kube-api-access-n8qwj\") pod \"payload-processing-e2e-ipp-a-c40fc3-5fb8bfbcff-tt9qf\" (UID: \"bf4909ee-383b-4d58-9232-127bb722e696\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c40fc3-5fb8bfbcff-tt9qf"
Oct 31 21:37:56.126122 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:56.126101    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf4909ee-383b-4d58-9232-127bb722e696-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-c40fc3-5fb8bfbcff-tt9qf\" (UID: \"bf4909ee-383b-4d58-9232-127bb722e696\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c40fc3-5fb8bfbcff-tt9qf"
Oct 31 21:37:56.313107 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:56.238621    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf4909ee-383b-4d58-9232-127bb722e696-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-c40fc3-5fb8bfbcff-tt9qf\" (UID: \"bf4909ee-383b-4d58-9232-127bb722e696\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c40fc3-5fb8bfbcff-tt9qf"
Oct 31 21:37:56.313107 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:56.238872    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8qwj\" (UniqueName: \"kubernetes.io/projected/bf4909ee-383b-4d58-9232-127bb722e696-kube-api-access-n8qwj\") pod \"payload-processing-e2e-ipp-a-c40fc3-5fb8bfbcff-tt9qf\" (UID: \"bf4909ee-383b-4d58-9232-127bb722e696\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c40fc3-5fb8bfbcff-tt9qf"
Oct 31 21:37:56.313107 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:56.240174    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf4909ee-383b-4d58-9232-127bb722e696-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-c40fc3-5fb8bfbcff-tt9qf\" (UID: \"bf4909ee-383b-4d58-9232-127bb722e696\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c40fc3-5fb8bfbcff-tt9qf"
Oct 31 21:37:56.364048 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:56.347852    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8qwj\" (UniqueName: \"kubernetes.io/projected/bf4909ee-383b-4d58-9232-127bb722e696-kube-api-access-n8qwj\") pod \"payload-processing-e2e-ipp-a-c40fc3-5fb8bfbcff-tt9qf\" (UID: \"bf4909ee-383b-4d58-9232-127bb722e696\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-c40fc3-5fb8bfbcff-tt9qf"
Oct 31 21:37:56.424004 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:56.419988    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:56.441309 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:56.441027    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-c40fc3-5fb8bfbcff-tt9qf"
Oct 31 21:37:56.520082 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:56.520050    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c40fc3-584b94f575-mkpjp"]
Oct 31 21:37:56.759089 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:56.759060    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c40fc3-5fb8bfbcff-tt9qf"]
Oct 31 21:37:56.759669 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:37:56.759646    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf4909ee_383b_4d58_9232_127bb722e696.slice/crio-713749d23bf55238b211cc8aa069abbec2f96072747ce4af75154f56e4d77e54 WatchSource:0}: Error finding container 713749d23bf55238b211cc8aa069abbec2f96072747ce4af75154f56e4d77e54: Status 404 returned error can't find the container with id 713749d23bf55238b211cc8aa069abbec2f96072747ce4af75154f56e4d77e54
Oct 31 21:37:57.113889 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:57.113838    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-c40fc3-5fb8bfbcff-tt9qf" event={"ID":"bf4909ee-383b-4d58-9232-127bb722e696","Type":"ContainerStarted","Data":"608e420af5e8d71055e761fbe59089691e68b2c6f209b0550d9db2730f125c02"}
Oct 31 21:37:57.114402 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:57.113911    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-c40fc3-5fb8bfbcff-tt9qf" event={"ID":"bf4909ee-383b-4d58-9232-127bb722e696","Type":"ContainerStarted","Data":"713749d23bf55238b211cc8aa069abbec2f96072747ce4af75154f56e4d77e54"}
Oct 31 21:37:57.114402 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:57.113975    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-c40fc3-5fb8bfbcff-tt9qf"
Oct 31 21:37:57.115610 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:57.115584    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c40fc3-584b94f575-mkpjp" event={"ID":"387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8","Type":"ContainerStarted","Data":"8b97581da762a0ade43be2faf287873f4229d927a24d7819e4feab5960d58a17"}
Oct 31 21:37:57.115734 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:57.115633    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c40fc3-584b94f575-mkpjp" event={"ID":"387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8","Type":"ContainerStarted","Data":"bb8ea6d3a0d20ee7aab77528c1ebb0ffe96bfc18f247152343ef1931969ef696"}
Oct 31 21:37:57.115734 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:57.115669    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c40fc3-584b94f575-mkpjp"
Oct 31 21:37:57.160557 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:57.160505    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-c40fc3-5fb8bfbcff-tt9qf" podStartSLOduration=2.160489315 podStartE2EDuration="2.160489315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:37:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:37:57.159048894 +0000 UTC m=+1317.075358337" watchObservedRunningTime="2026-10-31 21:37:57.160489315 +0000 UTC m=+1317.076798741"
Oct 31 21:37:57.203491 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:57.203433    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c40fc3-584b94f575-mkpjp" podStartSLOduration=2.203419522 podStartE2EDuration="2.203419522s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:37:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:37:57.202441805 +0000 UTC m=+1317.118751235" watchObservedRunningTime="2026-10-31 21:37:57.203419522 +0000 UTC m=+1317.119728951"
Oct 31 21:37:58.214549 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:58.214514    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:58.324848 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:58.324810    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:58.453157 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:58.453116    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:37:58.455953 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:58.455920    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-89d4a2-68bdd88d7-98td5"]
Oct 31 21:37:58.460363 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:58.460343    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-89d4a2-68bdd88d7-98td5"
Oct 31 21:37:58.480755 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:58.480710    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj8hg\" (UniqueName: \"kubernetes.io/projected/e0e99637-6888-4712-8a54-1f0627238131-kube-api-access-qj8hg\") pod \"payload-pre-processing-e2e-ipp-a-89d4a2-68bdd88d7-98td5\" (UID: \"e0e99637-6888-4712-8a54-1f0627238131\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-89d4a2-68bdd88d7-98td5"
Oct 31 21:37:58.480925 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:58.480757    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0e99637-6888-4712-8a54-1f0627238131-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-89d4a2-68bdd88d7-98td5\" (UID: \"e0e99637-6888-4712-8a54-1f0627238131\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-89d4a2-68bdd88d7-98td5"
Oct 31 21:37:58.482599 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:58.482574    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-89d4a2-68bdd88d7-98td5"]
Oct 31 21:37:58.490602 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:58.490577    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-89d4a2\""
Oct 31 21:37:58.490774 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:58.490614    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-89d4a2-dockercfg-r66xk\""
Oct 31 21:37:58.593996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:58.586115    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj8hg\" (UniqueName: \"kubernetes.io/projected/e0e99637-6888-4712-8a54-1f0627238131-kube-api-access-qj8hg\") pod \"payload-pre-processing-e2e-ipp-a-89d4a2-68bdd88d7-98td5\" (UID: \"e0e99637-6888-4712-8a54-1f0627238131\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-89d4a2-68bdd88d7-98td5"
Oct 31 21:37:58.593996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:58.586172    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0e99637-6888-4712-8a54-1f0627238131-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-89d4a2-68bdd88d7-98td5\" (UID: \"e0e99637-6888-4712-8a54-1f0627238131\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-89d4a2-68bdd88d7-98td5"
Oct 31 21:37:58.593996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:58.587298    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0e99637-6888-4712-8a54-1f0627238131-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-89d4a2-68bdd88d7-98td5\" (UID: \"e0e99637-6888-4712-8a54-1f0627238131\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-89d4a2-68bdd88d7-98td5"
Oct 31 21:37:58.613962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:58.608738    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-89d4a2-654f669f45-f4mvn"]
Oct 31 21:37:58.687966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:58.675409    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-89d4a2-654f669f45-f4mvn"
Oct 31 21:37:58.687966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:58.677873    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-89d4a2-654f669f45-f4mvn"]
Oct 31 21:37:58.693366 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:58.692675    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj8hg\" (UniqueName: \"kubernetes.io/projected/e0e99637-6888-4712-8a54-1f0627238131-kube-api-access-qj8hg\") pod \"payload-pre-processing-e2e-ipp-a-89d4a2-68bdd88d7-98td5\" (UID: \"e0e99637-6888-4712-8a54-1f0627238131\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-89d4a2-68bdd88d7-98td5"
Oct 31 21:37:58.785968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:58.783470    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-89d4a2-68bdd88d7-98td5"
Oct 31 21:37:58.825996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:58.825492    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4798f66b-8066-4794-8cbf-cbca86e87e3f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-89d4a2-654f669f45-f4mvn\" (UID: \"4798f66b-8066-4794-8cbf-cbca86e87e3f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-89d4a2-654f669f45-f4mvn"
Oct 31 21:37:58.825996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:58.825767    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4x52\" (UniqueName: \"kubernetes.io/projected/4798f66b-8066-4794-8cbf-cbca86e87e3f-kube-api-access-x4x52\") pod \"payload-processing-e2e-ipp-a-89d4a2-654f669f45-f4mvn\" (UID: \"4798f66b-8066-4794-8cbf-cbca86e87e3f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-89d4a2-654f669f45-f4mvn"
Oct 31 21:37:58.934330 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:58.932696    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4x52\" (UniqueName: \"kubernetes.io/projected/4798f66b-8066-4794-8cbf-cbca86e87e3f-kube-api-access-x4x52\") pod \"payload-processing-e2e-ipp-a-89d4a2-654f669f45-f4mvn\" (UID: \"4798f66b-8066-4794-8cbf-cbca86e87e3f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-89d4a2-654f669f45-f4mvn"
Oct 31 21:37:58.934330 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:58.932882    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4798f66b-8066-4794-8cbf-cbca86e87e3f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-89d4a2-654f669f45-f4mvn\" (UID: \"4798f66b-8066-4794-8cbf-cbca86e87e3f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-89d4a2-654f669f45-f4mvn"
Oct 31 21:37:58.938548 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:58.937906    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4798f66b-8066-4794-8cbf-cbca86e87e3f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-89d4a2-654f669f45-f4mvn\" (UID: \"4798f66b-8066-4794-8cbf-cbca86e87e3f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-89d4a2-654f669f45-f4mvn"
Oct 31 21:37:58.965745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:58.965712    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4x52\" (UniqueName: \"kubernetes.io/projected/4798f66b-8066-4794-8cbf-cbca86e87e3f-kube-api-access-x4x52\") pod \"payload-processing-e2e-ipp-a-89d4a2-654f669f45-f4mvn\" (UID: \"4798f66b-8066-4794-8cbf-cbca86e87e3f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-89d4a2-654f669f45-f4mvn"
Oct 31 21:37:59.147174 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:59.147092    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-89d4a2-654f669f45-f4mvn"
Oct 31 21:37:59.218267 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:59.217572    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-89d4a2-68bdd88d7-98td5"]
Oct 31 21:37:59.222033 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:37:59.221993    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0e99637_6888_4712_8a54_1f0627238131.slice/crio-71e18705925484a9fb21d814eeff70573425f27b9457a2b5c9eb8b7cd2d93d9f WatchSource:0}: Error finding container 71e18705925484a9fb21d814eeff70573425f27b9457a2b5c9eb8b7cd2d93d9f: Status 404 returned error can't find the container with id 71e18705925484a9fb21d814eeff70573425f27b9457a2b5c9eb8b7cd2d93d9f
Oct 31 21:37:59.453980 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:37:59.453898    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4798f66b_8066_4794_8cbf_cbca86e87e3f.slice/crio-81e489269efef3dacee50bd2538bf398aa5a291ca1d433afc81f0127313f77c5 WatchSource:0}: Error finding container 81e489269efef3dacee50bd2538bf398aa5a291ca1d433afc81f0127313f77c5: Status 404 returned error can't find the container with id 81e489269efef3dacee50bd2538bf398aa5a291ca1d433afc81f0127313f77c5
Oct 31 21:37:59.454413 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:37:59.454390    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-89d4a2-654f669f45-f4mvn"]
Oct 31 21:38:00.017806 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:00.017773    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:00.174032 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:00.173993    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-89d4a2-654f669f45-f4mvn" event={"ID":"4798f66b-8066-4794-8cbf-cbca86e87e3f","Type":"ContainerStarted","Data":"6e5c6be38cc1723265d2aa8682fbf6b98588f207cad7f99b1d51f2f21667a641"}
Oct 31 21:38:00.174237 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:00.174053    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-89d4a2-654f669f45-f4mvn" event={"ID":"4798f66b-8066-4794-8cbf-cbca86e87e3f","Type":"ContainerStarted","Data":"81e489269efef3dacee50bd2538bf398aa5a291ca1d433afc81f0127313f77c5"}
Oct 31 21:38:00.174237 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:00.174083    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-89d4a2-654f669f45-f4mvn"
Oct 31 21:38:00.175877 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:00.175844    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-89d4a2-68bdd88d7-98td5" event={"ID":"e0e99637-6888-4712-8a54-1f0627238131","Type":"ContainerStarted","Data":"5a09c04a68e791a703ad5c6fb56fb220d1a8e0186b4e783190a5ca3dd2da7598"}
Oct 31 21:38:00.176027 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:00.175898    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-89d4a2-68bdd88d7-98td5" event={"ID":"e0e99637-6888-4712-8a54-1f0627238131","Type":"ContainerStarted","Data":"71e18705925484a9fb21d814eeff70573425f27b9457a2b5c9eb8b7cd2d93d9f"}
Oct 31 21:38:00.176027 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:00.175974    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-89d4a2-68bdd88d7-98td5"
Oct 31 21:38:00.227127 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:00.227068    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-89d4a2-654f669f45-f4mvn" podStartSLOduration=2.227050161 podStartE2EDuration="2.227050161s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:37:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:38:00.223010207 +0000 UTC m=+1320.139319637" watchObservedRunningTime="2026-10-31 21:38:00.227050161 +0000 UTC m=+1320.143359592"
Oct 31 21:38:00.276270 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:00.276161    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-89d4a2-68bdd88d7-98td5" podStartSLOduration=2.2761473309999998 podStartE2EDuration="2.276147331s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:37:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:38:00.274303859 +0000 UTC m=+1320.190613289" watchObservedRunningTime="2026-10-31 21:38:00.276147331 +0000 UTC m=+1320.192456787"
Oct 31 21:38:00.594869 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:00.594779    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:00.714340 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:00.712637    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:00.952049 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:00.951959    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:01.512656 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:01.512617    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:01.748589 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:01.748554    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:01.919222 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:01.919136    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:02.194047 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.193967    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"]
Oct 31 21:38:02.200026 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.199995    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:38:02.220313 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.220286    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-07afdd-openshift-default-dockercfg-296r4\""
Oct 31 21:38:02.237694 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.237657    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"]
Oct 31 21:38:02.252381 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.252349    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:02.354963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.342543    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vxwg\" (UniqueName: \"kubernetes.io/projected/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-kube-api-access-9vxwg\") pod \"e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") " pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:38:02.375966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.366646    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istio-envoy\") pod \"e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") " pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:38:02.387497 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.387337    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istio-podinfo\") pod \"e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") " pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:38:02.388221 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.388193    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istio-token\") pod \"e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") " pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:38:02.395037 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.395010    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-credential-socket\") pod \"e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") " pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:38:02.395186 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.395169    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istio-data\") pod \"e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") " pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:38:02.395454 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.395435    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-workload-socket\") pod \"e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") " pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:38:02.395587 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.395570    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-workload-certs\") pod \"e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") " pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:38:02.395767 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.395749    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istiod-ca-cert\") pod \"e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") " pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:38:02.425298 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.425250    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:02.508008 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.503916    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istio-podinfo\") pod \"e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") " pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:38:02.508008 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.504149    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istio-token\") pod \"e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") " pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:38:02.508008 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.504177    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-credential-socket\") pod \"e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") " pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:38:02.508008 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.504204    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istio-data\") pod \"e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") " pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:38:02.508008 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.504306    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-workload-socket\") pod \"e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") " pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:38:02.508008 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.504329    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-workload-certs\") pod \"e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") " pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:38:02.508008 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.504385    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istiod-ca-cert\") pod \"e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") " pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:38:02.508008 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.504435    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vxwg\" (UniqueName: \"kubernetes.io/projected/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-kube-api-access-9vxwg\") pod \"e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") " pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:38:02.508008 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.504475    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istio-envoy\") pod \"e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") " pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:38:02.508008 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.505414    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istio-data\") pod \"e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") " pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:38:02.508008 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.506011    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-workload-certs\") pod \"e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") " pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:38:02.508008 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.506417    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-workload-socket\") pod \"e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") " pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:38:02.508008 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.507051    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istiod-ca-cert\") pod \"e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") " pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:38:02.508008 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.507754    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-credential-socket\") pod \"e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") " pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:38:02.547383 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.536844    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istio-envoy\") pod \"e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") " pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:38:02.547383 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.541668    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istio-podinfo\") pod \"e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") " pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:38:02.579243 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.576719    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istio-token\") pod \"e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") " pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:38:02.630967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.630203    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vxwg\" (UniqueName: \"kubernetes.io/projected/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-kube-api-access-9vxwg\") pod \"e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") " pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:38:02.819137 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:02.819102    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:38:03.149305 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:03.149273    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"]
Oct 31 21:38:03.149442 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:38:03.149406    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d68aba6_92a7_4f62_bc8f_d1fd40f64ed9.slice/crio-990f43e0eef8c639fe47912524b28b8e9e69e80398ded7fef28269b1ae468bac WatchSource:0}: Error finding container 990f43e0eef8c639fe47912524b28b8e9e69e80398ded7fef28269b1ae468bac: Status 404 returned error can't find the container with id 990f43e0eef8c639fe47912524b28b8e9e69e80398ded7fef28269b1ae468bac
Oct 31 21:38:03.152210 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:03.152178    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:38:03.152317 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:03.152251    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:38:03.152317 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:03.152296    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:38:03.233482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:03.233452    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td" event={"ID":"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9","Type":"ContainerStarted","Data":"ec253cef5e4639cf518d7b9e22cfb51844f7860cee985b4d87e91546777fba30"}
Oct 31 21:38:03.233599 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:03.233501    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td" event={"ID":"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9","Type":"ContainerStarted","Data":"990f43e0eef8c639fe47912524b28b8e9e69e80398ded7fef28269b1ae468bac"}
Oct 31 21:38:03.294497 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:03.294436    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td" podStartSLOduration=1.294421131 podStartE2EDuration="1.294421131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:38:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:38:03.292357694 +0000 UTC m=+1323.208667128" watchObservedRunningTime="2026-10-31 21:38:03.294421131 +0000 UTC m=+1323.210730626"
Oct 31 21:38:03.822048 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:03.821920    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:38:03.888599 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:03.888529    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:04.111030 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:04.107995    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:04.224678 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:04.224580    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:04.830560 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:04.830526    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:38:05.253982 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:05.253870    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:38:05.254810 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:05.254788    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:38:05.788160 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:05.788121    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:05.906863 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:05.906825    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:06.037985 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:06.035469    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:07.550254 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:07.550210    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:07.659597 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:07.659550    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:08.194477 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:08.193598    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c40fc3-584b94f575-mkpjp"
Oct 31 21:38:08.195966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:08.194830    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-c40fc3-5fb8bfbcff-tt9qf"
Oct 31 21:38:08.252607 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:08.252571    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:10.193556 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:10.193520    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:10.306518 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:10.306474    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:11.328880 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:11.328848    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-89d4a2-68bdd88d7-98td5"
Oct 31 21:38:11.398026 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:11.393439    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-89d4a2-654f669f45-f4mvn"
Oct 31 21:38:11.917204 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:11.917169    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2"]
Oct 31 21:38:11.923226 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:11.923202    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2"
Oct 31 21:38:11.956981 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:11.953227    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-89d4a2-serving-cert\""
Oct 31 21:38:11.959467 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:11.959437    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2"]
Oct 31 21:38:11.993580 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:11.993545    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:12.026916 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:12.026876    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8fc85a9-ec28-4322-a395-cc68f6a65947-maas-api-tls\") pod \"maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2\" (UID: \"e8fc85a9-ec28-4322-a395-cc68f6a65947\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2"
Oct 31 21:38:12.027124 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:12.027071    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8fc85a9-ec28-4322-a395-cc68f6a65947-metrics-certs\") pod \"maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2\" (UID: \"e8fc85a9-ec28-4322-a395-cc68f6a65947\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2"
Oct 31 21:38:12.027227 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:12.027208    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rv4cm\" (UniqueName: \"kubernetes.io/projected/e8fc85a9-ec28-4322-a395-cc68f6a65947-kube-api-access-rv4cm\") pod \"maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2\" (UID: \"e8fc85a9-ec28-4322-a395-cc68f6a65947\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2"
Oct 31 21:38:12.027265 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:12.027242    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8fc85a9-ec28-4322-a395-cc68f6a65947-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2\" (UID: \"e8fc85a9-ec28-4322-a395-cc68f6a65947\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2"
Oct 31 21:38:12.136426 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:12.136382    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rv4cm\" (UniqueName: \"kubernetes.io/projected/e8fc85a9-ec28-4322-a395-cc68f6a65947-kube-api-access-rv4cm\") pod \"maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2\" (UID: \"e8fc85a9-ec28-4322-a395-cc68f6a65947\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2"
Oct 31 21:38:12.137299 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:12.137274    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8fc85a9-ec28-4322-a395-cc68f6a65947-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2\" (UID: \"e8fc85a9-ec28-4322-a395-cc68f6a65947\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2"
Oct 31 21:38:12.137636 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:12.137613    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8fc85a9-ec28-4322-a395-cc68f6a65947-maas-api-tls\") pod \"maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2\" (UID: \"e8fc85a9-ec28-4322-a395-cc68f6a65947\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2"
Oct 31 21:38:12.137903 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:12.137881    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8fc85a9-ec28-4322-a395-cc68f6a65947-metrics-certs\") pod \"maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2\" (UID: \"e8fc85a9-ec28-4322-a395-cc68f6a65947\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2"
Oct 31 21:38:12.138188 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:38:12.138170    4460 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-89d4a2-serving-cert: secret "maas-api-e2e-ipp-a-89d4a2-serving-cert" not found
Oct 31 21:38:12.138341 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:38:12.138327    4460 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8fc85a9-ec28-4322-a395-cc68f6a65947-maas-api-tls podName:e8fc85a9-ec28-4322-a395-cc68f6a65947 nodeName:}" failed. No retries permitted until 2026-10-31 21:38:12.638306645 +0000 UTC m=+1332.554616067 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e8fc85a9-ec28-4322-a395-cc68f6a65947-maas-api-tls") pod "maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2" (UID: "e8fc85a9-ec28-4322-a395-cc68f6a65947") : secret "maas-api-e2e-ipp-a-89d4a2-serving-cert" not found
Oct 31 21:38:12.138546 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:12.138077    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8fc85a9-ec28-4322-a395-cc68f6a65947-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2\" (UID: \"e8fc85a9-ec28-4322-a395-cc68f6a65947\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2"
Oct 31 21:38:12.138661 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:12.137190    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:12.156798 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:12.156763    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8fc85a9-ec28-4322-a395-cc68f6a65947-metrics-certs\") pod \"maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2\" (UID: \"e8fc85a9-ec28-4322-a395-cc68f6a65947\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2"
Oct 31 21:38:12.171098 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:12.170918    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rv4cm\" (UniqueName: \"kubernetes.io/projected/e8fc85a9-ec28-4322-a395-cc68f6a65947-kube-api-access-rv4cm\") pod \"maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2\" (UID: \"e8fc85a9-ec28-4322-a395-cc68f6a65947\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2"
Oct 31 21:38:12.248176 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:12.248127    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:12.378076 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:12.378027    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:12.649430 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:12.649375    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8fc85a9-ec28-4322-a395-cc68f6a65947-maas-api-tls\") pod \"maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2\" (UID: \"e8fc85a9-ec28-4322-a395-cc68f6a65947\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2"
Oct 31 21:38:12.652499 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:12.652469    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8fc85a9-ec28-4322-a395-cc68f6a65947-maas-api-tls\") pod \"maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2\" (UID: \"e8fc85a9-ec28-4322-a395-cc68f6a65947\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2"
Oct 31 21:38:12.796732 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:12.796693    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:12.862752 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:12.862713    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2"
Oct 31 21:38:12.933395 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:12.919228    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:13.081239 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:13.081018    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:13.536963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:13.527346    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2"]
Oct 31 21:38:13.972306 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:13.972205    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:14.109490 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:14.106307    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:14.235496 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:14.235453    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:14.644584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:14.644543    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2" event={"ID":"e8fc85a9-ec28-4322-a395-cc68f6a65947","Type":"ContainerStarted","Data":"f9fa14423d5896c7ce20f4d4857396fa8f079664ba42e83fd8e89b8e72d90396"}
Oct 31 21:38:14.644584 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:14.644603    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2" event={"ID":"e8fc85a9-ec28-4322-a395-cc68f6a65947","Type":"ContainerStarted","Data":"24539374cccbf63b8c55c52834fbbcd588ffe509b3a90d1435d518e6d08940e1"}
Oct 31 21:38:14.645240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:14.644842    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2"
Oct 31 21:38:14.701512 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:14.698191    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2" podStartSLOduration=3.314717757 podStartE2EDuration="3.698167653s" totalImagesPullingTime="383.449896ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:38:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:38:14.693815822 +0000 UTC m=+1334.610125255" watchObservedRunningTime="2026-10-31 21:38:14.698167653 +0000 UTC m=+1334.614477084"
Oct 31 21:38:15.736634 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:15.736593    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:16.662148 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:16.662114    4460 generic.go:356] "Generic (PLEG): container finished" podID="ae15a792-70d4-40c4-8222-5a81b6e885be" containerID="e5346db7c8cd8b6dc928e9fc6064bcce5d46b3cd488c92e68e7104ecba4b5107" exitCode=0
Oct 31 21:38:16.662369 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:16.662188    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6" event={"ID":"ae15a792-70d4-40c4-8222-5a81b6e885be","Type":"ContainerDied","Data":"e5346db7c8cd8b6dc928e9fc6064bcce5d46b3cd488c92e68e7104ecba4b5107"}
Oct 31 21:38:16.662698 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:16.662683    4460 scope.go:122] "RemoveContainer" containerID="e5346db7c8cd8b6dc928e9fc6064bcce5d46b3cd488c92e68e7104ecba4b5107"
Oct 31 21:38:16.664378 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:16.664348    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:38:16.664471 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:16.664418    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:38:16.664471 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:16.664451    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:38:16.832641 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:16.832606    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2"]
Oct 31 21:38:16.837539 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:16.837499    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2"
Oct 31 21:38:16.858423 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:16.858388    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w4-f1e791\"/\"default-dockercfg-zvjsx\""
Oct 31 21:38:16.859172 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:16.859148    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w4-f1e791\"/\"kube-root-ca.crt\""
Oct 31 21:38:16.859265 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:16.859227    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w4-f1e791\"/\"openshift-service-ca.crt\""
Oct 31 21:38:16.859373 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:16.859345    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w4-f1e791\"/\"tenantcfg-model-f1e791-kserve-self-signed-certs\""
Oct 31 21:38:16.861695 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:16.861664    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2"]
Oct 31 21:38:16.883558 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:16.883520    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d183766a-a405-445c-975a-06efb0a90fed-dshm\") pod \"tenantcfg-model-f1e791-kserve-758b579b8f-86xs2\" (UID: \"d183766a-a405-445c-975a-06efb0a90fed\") " pod="ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2"
Oct 31 21:38:16.884064 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:16.884025    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d183766a-a405-445c-975a-06efb0a90fed-tls-certs\") pod \"tenantcfg-model-f1e791-kserve-758b579b8f-86xs2\" (UID: \"d183766a-a405-445c-975a-06efb0a90fed\") " pod="ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2"
Oct 31 21:38:16.889408 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:16.885472    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d183766a-a405-445c-975a-06efb0a90fed-tmp-dir\") pod \"tenantcfg-model-f1e791-kserve-758b579b8f-86xs2\" (UID: \"d183766a-a405-445c-975a-06efb0a90fed\") " pod="ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2"
Oct 31 21:38:16.889408 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:16.885621    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9b7tb\" (UniqueName: \"kubernetes.io/projected/d183766a-a405-445c-975a-06efb0a90fed-kube-api-access-9b7tb\") pod \"tenantcfg-model-f1e791-kserve-758b579b8f-86xs2\" (UID: \"d183766a-a405-445c-975a-06efb0a90fed\") " pod="ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2"
Oct 31 21:38:16.889408 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:16.886011    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d183766a-a405-445c-975a-06efb0a90fed-home\") pod \"tenantcfg-model-f1e791-kserve-758b579b8f-86xs2\" (UID: \"d183766a-a405-445c-975a-06efb0a90fed\") " pod="ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2"
Oct 31 21:38:16.889408 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:16.886205    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d183766a-a405-445c-975a-06efb0a90fed-model-cache\") pod \"tenantcfg-model-f1e791-kserve-758b579b8f-86xs2\" (UID: \"d183766a-a405-445c-975a-06efb0a90fed\") " pod="ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2"
Oct 31 21:38:16.897987 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:16.897910    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:16.988056 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:16.987959    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d183766a-a405-445c-975a-06efb0a90fed-tls-certs\") pod \"tenantcfg-model-f1e791-kserve-758b579b8f-86xs2\" (UID: \"d183766a-a405-445c-975a-06efb0a90fed\") " pod="ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2"
Oct 31 21:38:16.988056 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:16.988017    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d183766a-a405-445c-975a-06efb0a90fed-tmp-dir\") pod \"tenantcfg-model-f1e791-kserve-758b579b8f-86xs2\" (UID: \"d183766a-a405-445c-975a-06efb0a90fed\") " pod="ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2"
Oct 31 21:38:16.988288 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:16.988079    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9b7tb\" (UniqueName: \"kubernetes.io/projected/d183766a-a405-445c-975a-06efb0a90fed-kube-api-access-9b7tb\") pod \"tenantcfg-model-f1e791-kserve-758b579b8f-86xs2\" (UID: \"d183766a-a405-445c-975a-06efb0a90fed\") " pod="ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2"
Oct 31 21:38:16.988373 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:16.988353    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d183766a-a405-445c-975a-06efb0a90fed-home\") pod \"tenantcfg-model-f1e791-kserve-758b579b8f-86xs2\" (UID: \"d183766a-a405-445c-975a-06efb0a90fed\") " pod="ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2"
Oct 31 21:38:16.988469 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:16.988448    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d183766a-a405-445c-975a-06efb0a90fed-model-cache\") pod \"tenantcfg-model-f1e791-kserve-758b579b8f-86xs2\" (UID: \"d183766a-a405-445c-975a-06efb0a90fed\") " pod="ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2"
Oct 31 21:38:16.988622 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:16.988518    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d183766a-a405-445c-975a-06efb0a90fed-dshm\") pod \"tenantcfg-model-f1e791-kserve-758b579b8f-86xs2\" (UID: \"d183766a-a405-445c-975a-06efb0a90fed\") " pod="ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2"
Oct 31 21:38:16.988622 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:16.988551    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d183766a-a405-445c-975a-06efb0a90fed-tmp-dir\") pod \"tenantcfg-model-f1e791-kserve-758b579b8f-86xs2\" (UID: \"d183766a-a405-445c-975a-06efb0a90fed\") " pod="ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2"
Oct 31 21:38:16.988791 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:16.988770    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d183766a-a405-445c-975a-06efb0a90fed-home\") pod \"tenantcfg-model-f1e791-kserve-758b579b8f-86xs2\" (UID: \"d183766a-a405-445c-975a-06efb0a90fed\") " pod="ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2"
Oct 31 21:38:16.988861 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:16.988843    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d183766a-a405-445c-975a-06efb0a90fed-model-cache\") pod \"tenantcfg-model-f1e791-kserve-758b579b8f-86xs2\" (UID: \"d183766a-a405-445c-975a-06efb0a90fed\") " pod="ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2"
Oct 31 21:38:16.990657 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:16.990635    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d183766a-a405-445c-975a-06efb0a90fed-dshm\") pod \"tenantcfg-model-f1e791-kserve-758b579b8f-86xs2\" (UID: \"d183766a-a405-445c-975a-06efb0a90fed\") " pod="ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2"
Oct 31 21:38:16.990743 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:16.990715    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d183766a-a405-445c-975a-06efb0a90fed-tls-certs\") pod \"tenantcfg-model-f1e791-kserve-758b579b8f-86xs2\" (UID: \"d183766a-a405-445c-975a-06efb0a90fed\") " pod="ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2"
Oct 31 21:38:17.014114 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:17.014080    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9b7tb\" (UniqueName: \"kubernetes.io/projected/d183766a-a405-445c-975a-06efb0a90fed-kube-api-access-9b7tb\") pod \"tenantcfg-model-f1e791-kserve-758b579b8f-86xs2\" (UID: \"d183766a-a405-445c-975a-06efb0a90fed\") " pod="ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2"
Oct 31 21:38:17.167967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:17.167297    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2"
Oct 31 21:38:17.599024 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:38:17.595693    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd183766a_a405_445c_975a_06efb0a90fed.slice/crio-d9f13d48e8500fe665b640530b08d7206cedef0c9b647d99f99c7cbb9393b71e WatchSource:0}: Error finding container d9f13d48e8500fe665b640530b08d7206cedef0c9b647d99f99c7cbb9393b71e: Status 404 returned error can't find the container with id d9f13d48e8500fe665b640530b08d7206cedef0c9b647d99f99c7cbb9393b71e
Oct 31 21:38:17.599024 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:17.597620    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2"]
Oct 31 21:38:17.670068 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:17.670033    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:17.673806 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:17.673780    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6" event={"ID":"ae15a792-70d4-40c4-8222-5a81b6e885be","Type":"ContainerStarted","Data":"409dccab454a42ff858e8d77db1b0726d923c9de0d0f8239eac5d7201c917ad3"}
Oct 31 21:38:17.675219 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:17.675197    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2" event={"ID":"d183766a-a405-445c-975a-06efb0a90fed","Type":"ContainerStarted","Data":"d9f13d48e8500fe665b640530b08d7206cedef0c9b647d99f99c7cbb9393b71e"}
Oct 31 21:38:18.186776 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:18.186731    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:18.346981 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:18.346903    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:18.479815 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:18.479720    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:38:18.479992 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:18.479868    4460 patch_prober.go:29] interesting pod/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused" start-of-body=
Oct 31 21:38:18.479992 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:18.479917    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6" podUID="ae15a792-70d4-40c4-8222-5a81b6e885be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused"
Oct 31 21:38:18.685411 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:18.685372    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2" event={"ID":"d183766a-a405-445c-975a-06efb0a90fed","Type":"ContainerStarted","Data":"9339962abf5d19decca54eb788c8dcc078e4e3d634b1f92652f73b0a7c951523"}
Oct 31 21:38:18.747175 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:18.747051    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2" podStartSLOduration=2.7470307309999997 podStartE2EDuration="2.747030731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:38:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:38:18.746856226 +0000 UTC m=+1338.663165653" watchObservedRunningTime="2026-10-31 21:38:18.747030731 +0000 UTC m=+1338.663340163"
Oct 31 21:38:19.170927 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:19.170891    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2"
Oct 31 21:38:19.480152 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:19.480065    4460 patch_prober.go:29] interesting pod/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused" start-of-body=
Oct 31 21:38:19.480152 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:19.480123    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6" podUID="ae15a792-70d4-40c4-8222-5a81b6e885be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused"
Oct 31 21:38:19.617710 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:19.617674    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:19.959933 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:19.959897    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:20.074410 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:20.074375    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:20.667960 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:20.665273    4460 patch_prober.go:29] interesting pod/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused" start-of-body=
Oct 31 21:38:20.667960 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:20.665353    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6" podUID="ae15a792-70d4-40c4-8222-5a81b6e885be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused"
Oct 31 21:38:20.714019 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:20.712086    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2"
Oct 31 21:38:21.457517 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:21.457477    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:21.473693 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:38:21.473656    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:38:21.479450 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:21.479415    4460 patch_prober.go:29] interesting pod/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused" start-of-body=
Oct 31 21:38:21.479615 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:21.479486    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6" podUID="ae15a792-70d4-40c4-8222-5a81b6e885be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused"
Oct 31 21:38:22.479641 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:22.479600    4460 patch_prober.go:29] interesting pod/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused" start-of-body=
Oct 31 21:38:22.480104 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:22.479665    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6" podUID="ae15a792-70d4-40c4-8222-5a81b6e885be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.101:15021/healthz/ready\": dial tcp 10.133.0.101:15021: connect: connection refused"
Oct 31 21:38:23.239988 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:23.239953    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:23.484119 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:23.484087    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:38:23.484570 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:23.484557    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:38:23.485433 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:23.485411    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 21:38:25.033159 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:25.033093    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:26.845955 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:26.845900    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:26.984318 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:26.984273    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:27.171848 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:27.171772    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2"
Oct 31 21:38:27.184180 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:27.184153    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2"
Oct 31 21:38:27.855999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:27.855921    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2"
Oct 31 21:38:28.649500 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:28.649456    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:29.409446 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:29.409412    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:38:29.604207 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:29.604166    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:29.771977 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:29.762564    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:30.649990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:30.649925    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:31.945952 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:31.945909    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:32.195413 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:32.195377    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:33.158933 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:33.158896    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:35.076865 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:35.076824    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:36.282622 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:36.282586    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:37.072400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:37.072366    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:38.839452 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:38.839419    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:39.574690 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:39.574655    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c40fc3-5fb8bfbcff-tt9qf"]
Oct 31 21:38:39.648530 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:39.648495    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c40fc3-584b94f575-mkpjp"]
Oct 31 21:38:39.757812 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:39.757778    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:39.925856 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:39.925767    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:40.698100 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:40.696154    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:40.813475 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:40.813435    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:40.963066 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:40.962066    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:41.276059 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:41.276013    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:41.469673 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:41.469638    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:41.971439 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:41.971399    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-89d4a2-654f669f45-f4mvn"]
Oct 31 21:38:42.008473 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:42.008434    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-89d4a2-68bdd88d7-98td5"]
Oct 31 21:38:42.156890 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:42.156834    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:42.313361 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:42.313311    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:42.988789 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:42.988750    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:44.575486 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:44.575432    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-c40fc3-5fb8bfbcff-tt9qf" podUID="bf4909ee-383b-4d58-9232-127bb722e696" containerName="payload-processing" containerID="cri-o://608e420af5e8d71055e761fbe59089691e68b2c6f209b0550d9db2730f125c02" gracePeriod=25
Oct 31 21:38:44.649740 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:44.649691    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c40fc3-584b94f575-mkpjp" podUID="387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8" containerName="payload-pre-processing" containerID="cri-o://8b97581da762a0ade43be2faf287873f4229d927a24d7819e4feab5960d58a17" gracePeriod=25
Oct 31 21:38:44.960313 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:44.960256    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:45.075256 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.075218    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-c40fc3-5fb8bfbcff-tt9qf"
Oct 31 21:38:45.101974 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.099230    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bf4909ee-383b-4d58-9232-127bb722e696-kube-api-access-n8qwj\" (UniqueName: \"kubernetes.io/projected/bf4909ee-383b-4d58-9232-127bb722e696-kube-api-access-n8qwj\") pod \"bf4909ee-383b-4d58-9232-127bb722e696\" (UID: \"bf4909ee-383b-4d58-9232-127bb722e696\") "
Oct 31 21:38:45.101974 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.099480    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bf4909ee-383b-4d58-9232-127bb722e696-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf4909ee-383b-4d58-9232-127bb722e696-plugins-config-volume\") pod \"bf4909ee-383b-4d58-9232-127bb722e696\" (UID: \"bf4909ee-383b-4d58-9232-127bb722e696\") "
Oct 31 21:38:45.120974 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.120870    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf4909ee-383b-4d58-9232-127bb722e696-plugins-config-volume" pod "bf4909ee-383b-4d58-9232-127bb722e696" (UID: "bf4909ee-383b-4d58-9232-127bb722e696"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:38:45.121153 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.121106    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf4909ee-383b-4d58-9232-127bb722e696-kube-api-access-n8qwj" pod "bf4909ee-383b-4d58-9232-127bb722e696" (UID: "bf4909ee-383b-4d58-9232-127bb722e696"). InnerVolumeSpecName "kube-api-access-n8qwj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:38:45.122866 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.122836    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c40fc3-584b94f575-mkpjp"
Oct 31 21:38:45.139415 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.139374    4460 generic.go:356] "Generic (PLEG): container finished" podID="387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8" containerID="8b97581da762a0ade43be2faf287873f4229d927a24d7819e4feab5960d58a17" exitCode=0
Oct 31 21:38:45.139415 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.139430    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c40fc3-584b94f575-mkpjp"
Oct 31 21:38:45.139658 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.139471    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c40fc3-584b94f575-mkpjp" event={"ID":"387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8","Type":"ContainerDied","Data":"8b97581da762a0ade43be2faf287873f4229d927a24d7819e4feab5960d58a17"}
Oct 31 21:38:45.139658 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.139545    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c40fc3-584b94f575-mkpjp" event={"ID":"387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8","Type":"ContainerDied","Data":"bb8ea6d3a0d20ee7aab77528c1ebb0ffe96bfc18f247152343ef1931969ef696"}
Oct 31 21:38:45.139658 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.139577    4460 scope.go:122] "RemoveContainer" containerID="8b97581da762a0ade43be2faf287873f4229d927a24d7819e4feab5960d58a17"
Oct 31 21:38:45.141026 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.141000    4460 generic.go:356] "Generic (PLEG): container finished" podID="bf4909ee-383b-4d58-9232-127bb722e696" containerID="608e420af5e8d71055e761fbe59089691e68b2c6f209b0550d9db2730f125c02" exitCode=0
Oct 31 21:38:45.141150 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.141038    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-c40fc3-5fb8bfbcff-tt9qf" event={"ID":"bf4909ee-383b-4d58-9232-127bb722e696","Type":"ContainerDied","Data":"608e420af5e8d71055e761fbe59089691e68b2c6f209b0550d9db2730f125c02"}
Oct 31 21:38:45.141150 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.141066    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-c40fc3-5fb8bfbcff-tt9qf"
Oct 31 21:38:45.141150 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.141100    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-c40fc3-5fb8bfbcff-tt9qf" event={"ID":"bf4909ee-383b-4d58-9232-127bb722e696","Type":"ContainerDied","Data":"713749d23bf55238b211cc8aa069abbec2f96072747ce4af75154f56e4d77e54"}
Oct 31 21:38:45.156155 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.156119    4460 scope.go:122] "RemoveContainer" containerID="8b97581da762a0ade43be2faf287873f4229d927a24d7819e4feab5960d58a17"
Oct 31 21:38:45.156553 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:38:45.156523    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b97581da762a0ade43be2faf287873f4229d927a24d7819e4feab5960d58a17\": container with ID starting with 8b97581da762a0ade43be2faf287873f4229d927a24d7819e4feab5960d58a17 not found: ID does not exist" containerID="8b97581da762a0ade43be2faf287873f4229d927a24d7819e4feab5960d58a17"
Oct 31 21:38:45.156633 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.156568    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b97581da762a0ade43be2faf287873f4229d927a24d7819e4feab5960d58a17"} err="failed to get container status \"8b97581da762a0ade43be2faf287873f4229d927a24d7819e4feab5960d58a17\": rpc error: code = NotFound desc = could not find container \"8b97581da762a0ade43be2faf287873f4229d927a24d7819e4feab5960d58a17\": container with ID starting with 8b97581da762a0ade43be2faf287873f4229d927a24d7819e4feab5960d58a17 not found: ID does not exist"
Oct 31 21:38:45.156633 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.156596    4460 scope.go:122] "RemoveContainer" containerID="608e420af5e8d71055e761fbe59089691e68b2c6f209b0550d9db2730f125c02"
Oct 31 21:38:45.168949 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.168912    4460 scope.go:122] "RemoveContainer" containerID="608e420af5e8d71055e761fbe59089691e68b2c6f209b0550d9db2730f125c02"
Oct 31 21:38:45.169405 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:38:45.169377    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"608e420af5e8d71055e761fbe59089691e68b2c6f209b0550d9db2730f125c02\": container with ID starting with 608e420af5e8d71055e761fbe59089691e68b2c6f209b0550d9db2730f125c02 not found: ID does not exist" containerID="608e420af5e8d71055e761fbe59089691e68b2c6f209b0550d9db2730f125c02"
Oct 31 21:38:45.169491 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.169417    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"608e420af5e8d71055e761fbe59089691e68b2c6f209b0550d9db2730f125c02"} err="failed to get container status \"608e420af5e8d71055e761fbe59089691e68b2c6f209b0550d9db2730f125c02\": rpc error: code = NotFound desc = could not find container \"608e420af5e8d71055e761fbe59089691e68b2c6f209b0550d9db2730f125c02\": container with ID starting with 608e420af5e8d71055e761fbe59089691e68b2c6f209b0550d9db2730f125c02 not found: ID does not exist"
Oct 31 21:38:45.221726 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.221687    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8-plugins-config-volume\") pod \"387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8\" (UID: \"387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8\") "
Oct 31 21:38:45.222049 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.222025    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8-kube-api-access-dxzck\" (UniqueName: \"kubernetes.io/projected/387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8-kube-api-access-dxzck\") pod \"387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8\" (UID: \"387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8\") "
Oct 31 21:38:45.222258 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.222228    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8-plugins-config-volume" pod "387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8" (UID: "387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:38:45.222980 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.222818    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf4909ee-383b-4d58-9232-127bb722e696-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:38:45.222980 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.222847    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:38:45.222980 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.222862    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8qwj\" (UniqueName: \"kubernetes.io/projected/bf4909ee-383b-4d58-9232-127bb722e696-kube-api-access-n8qwj\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:38:45.224904 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.224870    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8-kube-api-access-dxzck" pod "387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8" (UID: "387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8"). InnerVolumeSpecName "kube-api-access-dxzck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:38:45.266496 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.266444    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c40fc3-5fb8bfbcff-tt9qf"]
Oct 31 21:38:45.270705 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.270671    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-c40fc3-5fb8bfbcff-tt9qf"]
Oct 31 21:38:45.324517 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.324482    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxzck\" (UniqueName: \"kubernetes.io/projected/387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8-kube-api-access-dxzck\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:38:45.495068 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.495026    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c40fc3-584b94f575-mkpjp"]
Oct 31 21:38:45.497163 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.497137    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c40fc3-584b94f575-mkpjp"]
Oct 31 21:38:45.499650 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.499615    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:45.622844 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.622767    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:45.748392 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:45.748310    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:46.227430 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:46.227396    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8" path="/var/lib/kubelet/pods/387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8/volumes"
Oct 31 21:38:46.227926 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:46.227910    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bf4909ee-383b-4d58-9232-127bb722e696" path="/var/lib/kubelet/pods/bf4909ee-383b-4d58-9232-127bb722e696/volumes"
Oct 31 21:38:46.294672 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:46.294638    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:46.972695 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:46.972650    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-89d4a2-654f669f45-f4mvn" podUID="4798f66b-8066-4794-8cbf-cbca86e87e3f" containerName="payload-processing" containerID="cri-o://6e5c6be38cc1723265d2aa8682fbf6b98588f207cad7f99b1d51f2f21667a641" gracePeriod=25
Oct 31 21:38:47.009035 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:47.008987    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-89d4a2-68bdd88d7-98td5" podUID="e0e99637-6888-4712-8a54-1f0627238131" containerName="payload-pre-processing" containerID="cri-o://5a09c04a68e791a703ad5c6fb56fb220d1a8e0186b4e783190a5ca3dd2da7598" gracePeriod=25
Oct 31 21:38:47.158655 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:47.158571    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:47.167681 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:47.167552    4460 generic.go:356] "Generic (PLEG): container finished" podID="e0e99637-6888-4712-8a54-1f0627238131" containerID="5a09c04a68e791a703ad5c6fb56fb220d1a8e0186b4e783190a5ca3dd2da7598" exitCode=0
Oct 31 21:38:47.167893 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:47.167727    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-89d4a2-68bdd88d7-98td5" event={"ID":"e0e99637-6888-4712-8a54-1f0627238131","Type":"ContainerDied","Data":"5a09c04a68e791a703ad5c6fb56fb220d1a8e0186b4e783190a5ca3dd2da7598"}
Oct 31 21:38:47.171448 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:47.169457    4460 generic.go:356] "Generic (PLEG): container finished" podID="4798f66b-8066-4794-8cbf-cbca86e87e3f" containerID="6e5c6be38cc1723265d2aa8682fbf6b98588f207cad7f99b1d51f2f21667a641" exitCode=0
Oct 31 21:38:47.171448 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:47.169582    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-89d4a2-654f669f45-f4mvn" event={"ID":"4798f66b-8066-4794-8cbf-cbca86e87e3f","Type":"ContainerDied","Data":"6e5c6be38cc1723265d2aa8682fbf6b98588f207cad7f99b1d51f2f21667a641"}
Oct 31 21:38:47.324612 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:47.324572    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:47.366506 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:47.366459    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-89d4a2-654f669f45-f4mvn"
Oct 31 21:38:47.383219 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:47.376627    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-89d4a2-68bdd88d7-98td5"
Oct 31 21:38:47.463693 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:47.463654    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:47.469736 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:47.469687    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e0e99637-6888-4712-8a54-1f0627238131-kube-api-access-qj8hg\" (UniqueName: \"kubernetes.io/projected/e0e99637-6888-4712-8a54-1f0627238131-kube-api-access-qj8hg\") pod \"e0e99637-6888-4712-8a54-1f0627238131\" (UID: \"e0e99637-6888-4712-8a54-1f0627238131\") "
Oct 31 21:38:47.469995 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:47.469884    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4798f66b-8066-4794-8cbf-cbca86e87e3f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4798f66b-8066-4794-8cbf-cbca86e87e3f-plugins-config-volume\") pod \"4798f66b-8066-4794-8cbf-cbca86e87e3f\" (UID: \"4798f66b-8066-4794-8cbf-cbca86e87e3f\") "
Oct 31 21:38:47.469995 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:47.469985    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e0e99637-6888-4712-8a54-1f0627238131-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0e99637-6888-4712-8a54-1f0627238131-plugins-config-volume\") pod \"e0e99637-6888-4712-8a54-1f0627238131\" (UID: \"e0e99637-6888-4712-8a54-1f0627238131\") "
Oct 31 21:38:47.470230 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:47.470208    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4798f66b-8066-4794-8cbf-cbca86e87e3f-kube-api-access-x4x52\" (UniqueName: \"kubernetes.io/projected/4798f66b-8066-4794-8cbf-cbca86e87e3f-kube-api-access-x4x52\") pod \"4798f66b-8066-4794-8cbf-cbca86e87e3f\" (UID: \"4798f66b-8066-4794-8cbf-cbca86e87e3f\") "
Oct 31 21:38:47.470507 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:47.470464    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0e99637-6888-4712-8a54-1f0627238131-plugins-config-volume" pod "e0e99637-6888-4712-8a54-1f0627238131" (UID: "e0e99637-6888-4712-8a54-1f0627238131"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:38:47.471353 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:47.471327    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0e99637-6888-4712-8a54-1f0627238131-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:38:47.471501 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:47.471473    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4798f66b-8066-4794-8cbf-cbca86e87e3f-plugins-config-volume" pod "4798f66b-8066-4794-8cbf-cbca86e87e3f" (UID: "4798f66b-8066-4794-8cbf-cbca86e87e3f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:38:47.472706 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:47.472678    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0e99637-6888-4712-8a54-1f0627238131-kube-api-access-qj8hg" pod "e0e99637-6888-4712-8a54-1f0627238131" (UID: "e0e99637-6888-4712-8a54-1f0627238131"). InnerVolumeSpecName "kube-api-access-qj8hg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:38:47.473033 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:47.473004    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4798f66b-8066-4794-8cbf-cbca86e87e3f-kube-api-access-x4x52" pod "4798f66b-8066-4794-8cbf-cbca86e87e3f" (UID: "4798f66b-8066-4794-8cbf-cbca86e87e3f"). InnerVolumeSpecName "kube-api-access-x4x52". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:38:47.573512 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:47.573416    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4x52\" (UniqueName: \"kubernetes.io/projected/4798f66b-8066-4794-8cbf-cbca86e87e3f-kube-api-access-x4x52\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:38:47.573512 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:47.573453    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qj8hg\" (UniqueName: \"kubernetes.io/projected/e0e99637-6888-4712-8a54-1f0627238131-kube-api-access-qj8hg\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:38:47.573512 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:47.573462    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4798f66b-8066-4794-8cbf-cbca86e87e3f-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:38:48.178802 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:48.178755    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-89d4a2-654f669f45-f4mvn" event={"ID":"4798f66b-8066-4794-8cbf-cbca86e87e3f","Type":"ContainerDied","Data":"81e489269efef3dacee50bd2538bf398aa5a291ca1d433afc81f0127313f77c5"}
Oct 31 21:38:48.178802 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:48.178786    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-89d4a2-654f669f45-f4mvn"
Oct 31 21:38:48.179379 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:48.178864    4460 scope.go:122] "RemoveContainer" containerID="6e5c6be38cc1723265d2aa8682fbf6b98588f207cad7f99b1d51f2f21667a641"
Oct 31 21:38:48.181073 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:48.181038    4460 generic.go:356] "Generic (PLEG): container finished" podID="b49ec684-cafd-4063-9ffa-c3884fbe7abd" containerID="c1073b04fdbe5c21ec9137e8bf45d09178688b7382975481c53333333a7d29b1" exitCode=0
Oct 31 21:38:48.181209 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:48.181112    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd" event={"ID":"b49ec684-cafd-4063-9ffa-c3884fbe7abd","Type":"ContainerDied","Data":"c1073b04fdbe5c21ec9137e8bf45d09178688b7382975481c53333333a7d29b1"}
Oct 31 21:38:48.181671 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:48.181648    4460 scope.go:122] "RemoveContainer" containerID="c1073b04fdbe5c21ec9137e8bf45d09178688b7382975481c53333333a7d29b1"
Oct 31 21:38:48.183116 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:48.183092    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-89d4a2-68bdd88d7-98td5" event={"ID":"e0e99637-6888-4712-8a54-1f0627238131","Type":"ContainerDied","Data":"71e18705925484a9fb21d814eeff70573425f27b9457a2b5c9eb8b7cd2d93d9f"}
Oct 31 21:38:48.183238 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:48.183180    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-89d4a2-68bdd88d7-98td5"
Oct 31 21:38:48.183535 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:48.183509    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:38:48.183614 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:48.183585    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:38:48.183678 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:48.183632    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:38:48.191453 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:48.191426    4460 scope.go:122] "RemoveContainer" containerID="5a09c04a68e791a703ad5c6fb56fb220d1a8e0186b4e783190a5ca3dd2da7598"
Oct 31 21:38:48.258863 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:48.258463    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-89d4a2-654f669f45-f4mvn"]
Oct 31 21:38:48.264378 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:48.264347    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-89d4a2-654f669f45-f4mvn"]
Oct 31 21:38:48.306897 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:48.306859    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-89d4a2-68bdd88d7-98td5"]
Oct 31 21:38:48.312558 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:48.312512    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-89d4a2-68bdd88d7-98td5"]
Oct 31 21:38:49.195521 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:49.195486    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd" event={"ID":"b49ec684-cafd-4063-9ffa-c3884fbe7abd","Type":"ContainerStarted","Data":"49a63fb4fe198f650749b8d65d097b9ff37d1864d463edc92bb93e5e9d8e3132"}
Oct 31 21:38:49.397084 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:49.396827    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:49.741962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:49.741006    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:38:49.741962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:49.741729    4460 patch_prober.go:29] interesting pod/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 31 21:38:49.741962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:49.741784    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd" podUID="b49ec684-cafd-4063-9ffa-c3884fbe7abd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 31 21:38:50.250409 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:50.250372    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4798f66b-8066-4794-8cbf-cbca86e87e3f" path="/var/lib/kubelet/pods/4798f66b-8066-4794-8cbf-cbca86e87e3f/volumes"
Oct 31 21:38:50.250932 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:50.250895    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e0e99637-6888-4712-8a54-1f0627238131" path="/var/lib/kubelet/pods/e0e99637-6888-4712-8a54-1f0627238131/volumes"
Oct 31 21:38:50.744002 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:50.743038    4460 patch_prober.go:29] interesting pod/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 31 21:38:50.744002 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:50.743109    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd" podUID="b49ec684-cafd-4063-9ffa-c3884fbe7abd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 31 21:38:51.247017 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:51.246958    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:51.474301 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:38:51.474259    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:38:51.740965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:51.738449    4460 patch_prober.go:29] interesting pod/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 31 21:38:51.740965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:51.738526    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd" podUID="b49ec684-cafd-4063-9ffa-c3884fbe7abd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 31 21:38:52.063703 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:52.063664    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:38:52.745802 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:52.745722    4460 patch_prober.go:29] interesting pod/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 31 21:38:52.745802 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:52.745780    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd" podUID="b49ec684-cafd-4063-9ffa-c3884fbe7abd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 31 21:38:52.953810 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:52.953771    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:53.740965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:53.734063    4460 patch_prober.go:29] interesting pod/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Oct 31 21:38:53.740965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:53.734126    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd" podUID="b49ec684-cafd-4063-9ffa-c3884fbe7abd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Oct 31 21:38:54.705632 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:54.705578    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:54.887036 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:54.886999    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:38:54.890092 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:54.890054    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:38:56.608569 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:56.608533    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:56.758720 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:56.758673    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:57.356326 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:57.356291    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:57.843837 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:57.843795    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:58.148019 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:58.147041    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:38:59.352559 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:38:59.352521    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:00.764787 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:00.764748    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:01.033020 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:01.032393    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:01.978692 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:01.978656    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:02.110683 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:02.110643    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:02.235622 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:02.235540    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:03.784063 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:03.784028    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:04.081389 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:04.081288    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:04.921898 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:04.921855    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:05.258137 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:05.258045    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:05.559094 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:05.559058    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:05.559310 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:05.559288    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"]
Oct 31 21:39:05.559508 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:05.559490    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm" podUID="f9d9464c-85ec-4323-81b6-7201345e9f07" containerName="istio-proxy" containerID="cri-o://bbb54219eb7004f0b8889566de9cd2a82fe6161b0d7931ba479dbc8e070227e7" gracePeriod=30
Oct 31 21:39:05.573009 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:05.572978    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"]
Oct 31 21:39:05.573232 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:05.573196    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz" podUID="2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa" containerName="istio-proxy" containerID="cri-o://90521185611ba3d14449a6f5008199c6b2753b54d2ad2d45b7f638308d41bd4c" gracePeriod=30
Oct 31 21:39:05.578526 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:05.578481    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"]
Oct 31 21:39:05.578813 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:05.578778    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh" podUID="ffaf1375-c974-4e1d-a8bd-b880d7ea6e96" containerName="istio-proxy" containerID="cri-o://b78878097670787314777a4ee6046b93b29fb04701737b8d65fcdc0f7f43eb8f" gracePeriod=30
Oct 31 21:39:07.457529 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:07.457488    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:09.123793 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:09.123753    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:09.698761 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:09.698723    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"]
Oct 31 21:39:10.072414 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:10.072376    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn" podUID="71d27085-7e15-413d-a20c-9e81d3a1123a" containerName="istio-proxy" containerID="cri-o://b9bed6215de842a8743f88fccf2aa0bd29f0509ca08abfbf35070caf0169a83d" gracePeriod=30
Oct 31 21:39:10.952270 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:10.951787    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:11.106208 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.106165    4460 generic.go:356] "Generic (PLEG): container finished" podID="f9d9464c-85ec-4323-81b6-7201345e9f07" containerID="bbb54219eb7004f0b8889566de9cd2a82fe6161b0d7931ba479dbc8e070227e7" exitCode=0
Oct 31 21:39:11.106350 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.106313    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm" event={"ID":"f9d9464c-85ec-4323-81b6-7201345e9f07","Type":"ContainerDied","Data":"bbb54219eb7004f0b8889566de9cd2a82fe6161b0d7931ba479dbc8e070227e7"}
Oct 31 21:39:11.108588 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.108560    4460 generic.go:356] "Generic (PLEG): container finished" podID="2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa" containerID="90521185611ba3d14449a6f5008199c6b2753b54d2ad2d45b7f638308d41bd4c" exitCode=0
Oct 31 21:39:11.108710 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.108630    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz" event={"ID":"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa","Type":"ContainerDied","Data":"90521185611ba3d14449a6f5008199c6b2753b54d2ad2d45b7f638308d41bd4c"}
Oct 31 21:39:11.110920 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.110893    4460 generic.go:356] "Generic (PLEG): container finished" podID="ffaf1375-c974-4e1d-a8bd-b880d7ea6e96" containerID="b78878097670787314777a4ee6046b93b29fb04701737b8d65fcdc0f7f43eb8f" exitCode=0
Oct 31 21:39:11.111081 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.110961    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh" event={"ID":"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96","Type":"ContainerDied","Data":"b78878097670787314777a4ee6046b93b29fb04701737b8d65fcdc0f7f43eb8f"}
Oct 31 21:39:11.206757 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.206674    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:39:11.249364 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.249340    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:39:11.252977 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.252933    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:39:11.319090 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.319057    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-credential-socket\") pod \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") "
Oct 31 21:39:11.319297 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.319106    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-credential-socket\") pod \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") "
Oct 31 21:39:11.319297 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.319150    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f9d9464c-85ec-4323-81b6-7201345e9f07-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9d9464c-85ec-4323-81b6-7201345e9f07-istio-podinfo\") pod \"f9d9464c-85ec-4323-81b6-7201345e9f07\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") "
Oct 31 21:39:11.319297 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.319203    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istio-token\" (UniqueName: \"kubernetes.io/projected/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istio-token\") pod \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") "
Oct 31 21:39:11.319297 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.319271    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istio-envoy\") pod \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") "
Oct 31 21:39:11.319521 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.319317    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istio-data\") pod \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") "
Oct 31 21:39:11.319521 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.319360    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istio-envoy\") pod \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") "
Oct 31 21:39:11.319521 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.319363    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-credential-socket" pod "2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa" (UID: "2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:39:11.319521 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.319389    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-workload-certs\") pod \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") "
Oct 31 21:39:11.319521 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.319445    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-credential-socket" pod "ffaf1375-c974-4e1d-a8bd-b880d7ea6e96" (UID: "ffaf1375-c974-4e1d-a8bd-b880d7ea6e96"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:39:11.319521 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.319479    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-workload-socket\") pod \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") "
Oct 31 21:39:11.319521 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.319518    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-workload-socket\") pod \"f9d9464c-85ec-4323-81b6-7201345e9f07\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") "
Oct 31 21:39:11.319879 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.319584    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istio-data\") pod \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") "
Oct 31 21:39:11.319879 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.319598    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-workload-certs" pod "2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa" (UID: "2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:39:11.319879 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.319658    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f9d9464c-85ec-4323-81b6-7201345e9f07-kube-api-access-mq9xh\" (UniqueName: \"kubernetes.io/projected/f9d9464c-85ec-4323-81b6-7201345e9f07-kube-api-access-mq9xh\") pod \"f9d9464c-85ec-4323-81b6-7201345e9f07\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") "
Oct 31 21:39:11.319879 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.319684    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istio-data" pod "2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa" (UID: "2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:39:11.319879 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.319702    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istio-podinfo\") pod \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") "
Oct 31 21:39:11.319879 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.319738    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-workload-certs\") pod \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") "
Oct 31 21:39:11.319879 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.319787    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-istio-envoy\") pod \"f9d9464c-85ec-4323-81b6-7201345e9f07\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") "
Oct 31 21:39:11.319879 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.319820    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istiod-ca-cert\") pod \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") "
Oct 31 21:39:11.319879 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.319836    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-workload-socket" pod "f9d9464c-85ec-4323-81b6-7201345e9f07" (UID: "f9d9464c-85ec-4323-81b6-7201345e9f07"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:39:11.319879 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.319849    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-credential-socket\") pod \"f9d9464c-85ec-4323-81b6-7201345e9f07\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") "
Oct 31 21:39:11.319879 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.319867    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istio-data" pod "ffaf1375-c974-4e1d-a8bd-b880d7ea6e96" (UID: "ffaf1375-c974-4e1d-a8bd-b880d7ea6e96"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:39:11.320427 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.319903    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-workload-certs\") pod \"f9d9464c-85ec-4323-81b6-7201345e9f07\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") "
Oct 31 21:39:11.320427 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.319980    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f9d9464c-85ec-4323-81b6-7201345e9f07-istio-token\" (UniqueName: \"kubernetes.io/projected/f9d9464c-85ec-4323-81b6-7201345e9f07-istio-token\") pod \"f9d9464c-85ec-4323-81b6-7201345e9f07\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") "
Oct 31 21:39:11.320427 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.320029    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istiod-ca-cert\") pod \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") "
Oct 31 21:39:11.320427 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.320056    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-workload-certs" pod "ffaf1375-c974-4e1d-a8bd-b880d7ea6e96" (UID: "ffaf1375-c974-4e1d-a8bd-b880d7ea6e96"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:39:11.320427 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.320120    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-istio-data\") pod \"f9d9464c-85ec-4323-81b6-7201345e9f07\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") "
Oct 31 21:39:11.320427 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.320156    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-kube-api-access-dmp2v\" (UniqueName: \"kubernetes.io/projected/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-kube-api-access-dmp2v\") pod \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") "
Oct 31 21:39:11.320427 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.320190    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-workload-socket" pod "ffaf1375-c974-4e1d-a8bd-b880d7ea6e96" (UID: "ffaf1375-c974-4e1d-a8bd-b880d7ea6e96"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:39:11.320427 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.320208    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istio-token\" (UniqueName: \"kubernetes.io/projected/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istio-token\") pod \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") "
Oct 31 21:39:11.320427 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.320328    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-workload-socket\") pod \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\" (UID: \"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa\") "
Oct 31 21:39:11.320427 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.320370    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istio-podinfo\") pod \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") "
Oct 31 21:39:11.320906 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.320448    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f9d9464c-85ec-4323-81b6-7201345e9f07-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9d9464c-85ec-4323-81b6-7201345e9f07-istiod-ca-cert\") pod \"f9d9464c-85ec-4323-81b6-7201345e9f07\" (UID: \"f9d9464c-85ec-4323-81b6-7201345e9f07\") "
Oct 31 21:39:11.320906 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.320504    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-kube-api-access-7ztxn\" (UniqueName: \"kubernetes.io/projected/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-kube-api-access-7ztxn\") pod \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\" (UID: \"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96\") "
Oct 31 21:39:11.322095 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.321142    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-istio-data" pod "f9d9464c-85ec-4323-81b6-7201345e9f07" (UID: "f9d9464c-85ec-4323-81b6-7201345e9f07"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:39:11.322095 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.321432    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-workload-certs" pod "f9d9464c-85ec-4323-81b6-7201345e9f07" (UID: "f9d9464c-85ec-4323-81b6-7201345e9f07"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:39:11.322095 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.321545    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:11.322095 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.321569    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:11.322095 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.321584    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:11.322095 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.321600    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:11.322095 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.321617    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:11.322095 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.321631    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:11.322095 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.321646    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:11.322095 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.321661    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:11.322095 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.321674    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:11.322095 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.321757    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istiod-ca-cert" pod "2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa" (UID: "2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:39:11.322095 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.322050    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-credential-socket" pod "f9d9464c-85ec-4323-81b6-7201345e9f07" (UID: "f9d9464c-85ec-4323-81b6-7201345e9f07"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:39:11.327320 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.323230    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istiod-ca-cert" pod "ffaf1375-c974-4e1d-a8bd-b880d7ea6e96" (UID: "ffaf1375-c974-4e1d-a8bd-b880d7ea6e96"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:39:11.327320 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.323641    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9d9464c-85ec-4323-81b6-7201345e9f07-istiod-ca-cert" pod "f9d9464c-85ec-4323-81b6-7201345e9f07" (UID: "f9d9464c-85ec-4323-81b6-7201345e9f07"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:39:11.327320 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.324303    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-kube-api-access-dmp2v" pod "2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa" (UID: "2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa"). InnerVolumeSpecName "kube-api-access-dmp2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:39:11.327320 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.326141    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istio-token" pod "2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa" (UID: "2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:39:11.327320 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.326453    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-workload-socket" pod "2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa" (UID: "2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:39:11.327320 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.327011    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-kube-api-access-7ztxn" pod "ffaf1375-c974-4e1d-a8bd-b880d7ea6e96" (UID: "ffaf1375-c974-4e1d-a8bd-b880d7ea6e96"). InnerVolumeSpecName "kube-api-access-7ztxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:39:11.329573 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.328067    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-istio-envoy" pod "f9d9464c-85ec-4323-81b6-7201345e9f07" (UID: "f9d9464c-85ec-4323-81b6-7201345e9f07"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:39:11.329573 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.328465    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9d9464c-85ec-4323-81b6-7201345e9f07-istio-token" pod "f9d9464c-85ec-4323-81b6-7201345e9f07" (UID: "f9d9464c-85ec-4323-81b6-7201345e9f07"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:39:11.329573 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.328546    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istio-podinfo" pod "ffaf1375-c974-4e1d-a8bd-b880d7ea6e96" (UID: "ffaf1375-c974-4e1d-a8bd-b880d7ea6e96"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:39:11.330332 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.330291    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istio-envoy" pod "2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa" (UID: "2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:39:11.330332 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.330294    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istio-podinfo" pod "2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa" (UID: "2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:39:11.330332 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.330314    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istio-envoy" pod "ffaf1375-c974-4e1d-a8bd-b880d7ea6e96" (UID: "ffaf1375-c974-4e1d-a8bd-b880d7ea6e96"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:39:11.330703 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.330667    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f9d9464c-85ec-4323-81b6-7201345e9f07-istio-podinfo" pod "f9d9464c-85ec-4323-81b6-7201345e9f07" (UID: "f9d9464c-85ec-4323-81b6-7201345e9f07"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:39:11.331096 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.331070    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9d9464c-85ec-4323-81b6-7201345e9f07-kube-api-access-mq9xh" pod "f9d9464c-85ec-4323-81b6-7201345e9f07" (UID: "f9d9464c-85ec-4323-81b6-7201345e9f07"). InnerVolumeSpecName "kube-api-access-mq9xh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:39:11.332558 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.332159    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istio-token" pod "ffaf1375-c974-4e1d-a8bd-b880d7ea6e96" (UID: "ffaf1375-c974-4e1d-a8bd-b880d7ea6e96"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:39:11.423308 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.423272    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mq9xh\" (UniqueName: \"kubernetes.io/projected/f9d9464c-85ec-4323-81b6-7201345e9f07-kube-api-access-mq9xh\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:11.423308 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.423305    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:11.423308 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.423316    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:11.423544 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.423325    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:11.423544 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.423335    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:11.423544 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.423344    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9d9464c-85ec-4323-81b6-7201345e9f07-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:11.423544 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.423352    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9d9464c-85ec-4323-81b6-7201345e9f07-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:11.423544 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.423362    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:11.423544 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.423371    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmp2v\" (UniqueName: \"kubernetes.io/projected/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-kube-api-access-dmp2v\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:11.423544 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.423379    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:11.423544 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.423387    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:11.423544 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.423395    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:11.423544 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.423404    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9d9464c-85ec-4323-81b6-7201345e9f07-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:11.423544 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.423412    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ztxn\" (UniqueName: \"kubernetes.io/projected/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-kube-api-access-7ztxn\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:11.423544 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.423421    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9d9464c-85ec-4323-81b6-7201345e9f07-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:11.423544 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.423430    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:11.423544 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.423440    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:11.423544 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:11.423449    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:12.119171 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:12.119136    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"
Oct 31 21:39:12.119171 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:12.119147    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm" event={"ID":"f9d9464c-85ec-4323-81b6-7201345e9f07","Type":"ContainerDied","Data":"74acce2f92d2b5d8ef9fabf530c9442d95af220008c63c92d258c8521afb816c"}
Oct 31 21:39:12.119719 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:12.119231    4460 scope.go:122] "RemoveContainer" containerID="bbb54219eb7004f0b8889566de9cd2a82fe6161b0d7931ba479dbc8e070227e7"
Oct 31 21:39:12.120747 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:12.120729    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"
Oct 31 21:39:12.120853 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:12.120771    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz" event={"ID":"2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa","Type":"ContainerDied","Data":"d45878df72b1da34423dd1922998cd9ad9d4713a33e3ccc813ad78907891ea28"}
Oct 31 21:39:12.122662 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:12.122638    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh" event={"ID":"ffaf1375-c974-4e1d-a8bd-b880d7ea6e96","Type":"ContainerDied","Data":"e9d2ff52d274093581f4e0694871afc5628dece656a812e7936c26f02ed55907"}
Oct 31 21:39:12.122760 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:12.122685    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"
Oct 31 21:39:12.132262 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:12.132237    4460 scope.go:122] "RemoveContainer" containerID="90521185611ba3d14449a6f5008199c6b2753b54d2ad2d45b7f638308d41bd4c"
Oct 31 21:39:12.142535 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:12.142515    4460 scope.go:122] "RemoveContainer" containerID="b78878097670787314777a4ee6046b93b29fb04701737b8d65fcdc0f7f43eb8f"
Oct 31 21:39:12.191934 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:12.191895    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"]
Oct 31 21:39:12.205342 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:12.205311    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-q5wrm"]
Oct 31 21:39:12.226733 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:12.226698    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f9d9464c-85ec-4323-81b6-7201345e9f07" path="/var/lib/kubelet/pods/f9d9464c-85ec-4323-81b6-7201345e9f07/volumes"
Oct 31 21:39:12.244304 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:12.244264    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"]
Oct 31 21:39:12.248785 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:12.248758    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kwcbh"]
Oct 31 21:39:12.296208 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:12.296178    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"]
Oct 31 21:39:12.299607 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:12.299570    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-kbgkz"]
Oct 31 21:39:12.929505 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:12.929410    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:13.071960 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:13.062523    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:14.244747 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:14.244709    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa" path="/var/lib/kubelet/pods/2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa/volumes"
Oct 31 21:39:14.246104 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:14.246076    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ffaf1375-c974-4e1d-a8bd-b880d7ea6e96" path="/var/lib/kubelet/pods/ffaf1375-c974-4e1d-a8bd-b880d7ea6e96/volumes"
Oct 31 21:39:14.386561 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:14.386524    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:14.501058 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:14.500925    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:14.609921 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:14.609888    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:14.831148 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:14.831108    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:15.253836 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:15.253804    4460 generic.go:356] "Generic (PLEG): container finished" podID="71d27085-7e15-413d-a20c-9e81d3a1123a" containerID="b9bed6215de842a8743f88fccf2aa0bd29f0509ca08abfbf35070caf0169a83d" exitCode=0
Oct 31 21:39:15.254260 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:15.253886    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn" event={"ID":"71d27085-7e15-413d-a20c-9e81d3a1123a","Type":"ContainerDied","Data":"b9bed6215de842a8743f88fccf2aa0bd29f0509ca08abfbf35070caf0169a83d"}
Oct 31 21:39:15.401764 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:15.401734    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:39:15.536085 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:15.536039    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/71d27085-7e15-413d-a20c-9e81d3a1123a-kube-api-access-96h56\" (UniqueName: \"kubernetes.io/projected/71d27085-7e15-413d-a20c-9e81d3a1123a-kube-api-access-96h56\") pod \"71d27085-7e15-413d-a20c-9e81d3a1123a\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") "
Oct 31 21:39:15.536284 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:15.536120    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-istio-envoy\") pod \"71d27085-7e15-413d-a20c-9e81d3a1123a\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") "
Oct 31 21:39:15.536284 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:15.536238    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-workload-certs\") pod \"71d27085-7e15-413d-a20c-9e81d3a1123a\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") "
Oct 31 21:39:15.536409 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:15.536318    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-workload-socket\") pod \"71d27085-7e15-413d-a20c-9e81d3a1123a\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") "
Oct 31 21:39:15.536494 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:15.536459    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/71d27085-7e15-413d-a20c-9e81d3a1123a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71d27085-7e15-413d-a20c-9e81d3a1123a-istio-podinfo\") pod \"71d27085-7e15-413d-a20c-9e81d3a1123a\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") "
Oct 31 21:39:15.536556 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:15.536523    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-workload-certs" pod "71d27085-7e15-413d-a20c-9e81d3a1123a" (UID: "71d27085-7e15-413d-a20c-9e81d3a1123a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:39:15.536618 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:15.536560    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/71d27085-7e15-413d-a20c-9e81d3a1123a-istio-token\" (UniqueName: \"kubernetes.io/projected/71d27085-7e15-413d-a20c-9e81d3a1123a-istio-token\") pod \"71d27085-7e15-413d-a20c-9e81d3a1123a\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") "
Oct 31 21:39:15.536618 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:15.536610    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-credential-socket\") pod \"71d27085-7e15-413d-a20c-9e81d3a1123a\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") "
Oct 31 21:39:15.536734 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:15.536561    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-workload-socket" pod "71d27085-7e15-413d-a20c-9e81d3a1123a" (UID: "71d27085-7e15-413d-a20c-9e81d3a1123a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:39:15.536734 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:15.536667    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/71d27085-7e15-413d-a20c-9e81d3a1123a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71d27085-7e15-413d-a20c-9e81d3a1123a-istiod-ca-cert\") pod \"71d27085-7e15-413d-a20c-9e81d3a1123a\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") "
Oct 31 21:39:15.536838 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:15.536777    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-istio-data\") pod \"71d27085-7e15-413d-a20c-9e81d3a1123a\" (UID: \"71d27085-7e15-413d-a20c-9e81d3a1123a\") "
Oct 31 21:39:15.536918 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:15.536888    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-credential-socket" pod "71d27085-7e15-413d-a20c-9e81d3a1123a" (UID: "71d27085-7e15-413d-a20c-9e81d3a1123a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:39:15.537202 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:15.537178    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-istio-data" pod "71d27085-7e15-413d-a20c-9e81d3a1123a" (UID: "71d27085-7e15-413d-a20c-9e81d3a1123a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:39:15.537278 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:15.537217    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71d27085-7e15-413d-a20c-9e81d3a1123a-istiod-ca-cert" pod "71d27085-7e15-413d-a20c-9e81d3a1123a" (UID: "71d27085-7e15-413d-a20c-9e81d3a1123a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:39:15.537528 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:15.537508    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:15.537570 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:15.537538    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:15.537570 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:15.537554    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:15.537636 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:15.537569    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:15.537636 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:15.537586    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71d27085-7e15-413d-a20c-9e81d3a1123a-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:15.538587 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:15.538564    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71d27085-7e15-413d-a20c-9e81d3a1123a-kube-api-access-96h56" pod "71d27085-7e15-413d-a20c-9e81d3a1123a" (UID: "71d27085-7e15-413d-a20c-9e81d3a1123a"). InnerVolumeSpecName "kube-api-access-96h56". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:39:15.538977 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:15.538918    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-istio-envoy" pod "71d27085-7e15-413d-a20c-9e81d3a1123a" (UID: "71d27085-7e15-413d-a20c-9e81d3a1123a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:39:15.539081 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:15.539058    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/71d27085-7e15-413d-a20c-9e81d3a1123a-istio-podinfo" pod "71d27085-7e15-413d-a20c-9e81d3a1123a" (UID: "71d27085-7e15-413d-a20c-9e81d3a1123a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:39:15.539126 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:15.539063    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71d27085-7e15-413d-a20c-9e81d3a1123a-istio-token" pod "71d27085-7e15-413d-a20c-9e81d3a1123a" (UID: "71d27085-7e15-413d-a20c-9e81d3a1123a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:39:15.638918 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:15.638806    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71d27085-7e15-413d-a20c-9e81d3a1123a-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:15.638918 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:15.638849    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71d27085-7e15-413d-a20c-9e81d3a1123a-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:15.638918 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:15.638863    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96h56\" (UniqueName: \"kubernetes.io/projected/71d27085-7e15-413d-a20c-9e81d3a1123a-kube-api-access-96h56\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:15.638918 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:15.638877    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71d27085-7e15-413d-a20c-9e81d3a1123a-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:39:16.263500 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:16.263456    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn" event={"ID":"71d27085-7e15-413d-a20c-9e81d3a1123a","Type":"ContainerDied","Data":"384c63b7f6c4d4e4e1e57b9939cb578b8cde96daaa569e6413032d10975cccd6"}
Oct 31 21:39:16.263982 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:16.263532    4460 scope.go:122] "RemoveContainer" containerID="b9bed6215de842a8743f88fccf2aa0bd29f0509ca08abfbf35070caf0169a83d"
Oct 31 21:39:16.263982 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:16.263471    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"
Oct 31 21:39:16.337448 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:16.337404    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"]
Oct 31 21:39:16.350606 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:16.350570    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-82wwn"]
Oct 31 21:39:16.641401 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:16.641366    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:16.774934 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:16.774082    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:16.904831 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:16.904733    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:17.370810 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.370777    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"]
Oct 31 21:39:17.371443 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.371427    4460 cpu_manager.go:414] "containerMap: removing container" podUID="bf4909ee-383b-4d58-9232-127bb722e696" containerName="payload-processing"
Oct 31 21:39:17.371443 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.371443    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bf4909ee-383b-4d58-9232-127bb722e696" containerName="payload-processing"
Oct 31 21:39:17.371572 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.371454    4460 cpu_manager.go:414] "containerMap: removing container" podUID="f9d9464c-85ec-4323-81b6-7201345e9f07" containerName="istio-proxy"
Oct 31 21:39:17.371572 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.371459    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f9d9464c-85ec-4323-81b6-7201345e9f07" containerName="istio-proxy"
Oct 31 21:39:17.371572 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.371470    4460 cpu_manager.go:414] "containerMap: removing container" podUID="71d27085-7e15-413d-a20c-9e81d3a1123a" containerName="istio-proxy"
Oct 31 21:39:17.371572 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.371475    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="71d27085-7e15-413d-a20c-9e81d3a1123a" containerName="istio-proxy"
Oct 31 21:39:17.371572 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.371488    4460 cpu_manager.go:414] "containerMap: removing container" podUID="2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa" containerName="istio-proxy"
Oct 31 21:39:17.371572 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.371493    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa" containerName="istio-proxy"
Oct 31 21:39:17.371572 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.371503    4460 cpu_manager.go:414] "containerMap: removing container" podUID="ffaf1375-c974-4e1d-a8bd-b880d7ea6e96" containerName="istio-proxy"
Oct 31 21:39:17.371572 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.371508    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ffaf1375-c974-4e1d-a8bd-b880d7ea6e96" containerName="istio-proxy"
Oct 31 21:39:17.371572 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.371519    4460 cpu_manager.go:414] "containerMap: removing container" podUID="4798f66b-8066-4794-8cbf-cbca86e87e3f" containerName="payload-processing"
Oct 31 21:39:17.371572 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.371523    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4798f66b-8066-4794-8cbf-cbca86e87e3f" containerName="payload-processing"
Oct 31 21:39:17.371572 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.371534    4460 cpu_manager.go:414] "containerMap: removing container" podUID="387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8" containerName="payload-pre-processing"
Oct 31 21:39:17.371572 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.371538    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8" containerName="payload-pre-processing"
Oct 31 21:39:17.371572 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.371560    4460 cpu_manager.go:414] "containerMap: removing container" podUID="e0e99637-6888-4712-8a54-1f0627238131" containerName="payload-pre-processing"
Oct 31 21:39:17.371572 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.371565    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e0e99637-6888-4712-8a54-1f0627238131" containerName="payload-pre-processing"
Oct 31 21:39:17.372355 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.371647    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="e0e99637-6888-4712-8a54-1f0627238131" containerName="payload-pre-processing"
Oct 31 21:39:17.372355 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.371659    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="bf4909ee-383b-4d58-9232-127bb722e696" containerName="payload-processing"
Oct 31 21:39:17.372355 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.371666    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="f9d9464c-85ec-4323-81b6-7201345e9f07" containerName="istio-proxy"
Oct 31 21:39:17.372355 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.371673    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="71d27085-7e15-413d-a20c-9e81d3a1123a" containerName="istio-proxy"
Oct 31 21:39:17.372355 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.371680    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b5ffb34-9d39-4ec9-b4f8-5ed2d90d6daa" containerName="istio-proxy"
Oct 31 21:39:17.372355 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.371685    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="ffaf1375-c974-4e1d-a8bd-b880d7ea6e96" containerName="istio-proxy"
Oct 31 21:39:17.372355 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.371691    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="4798f66b-8066-4794-8cbf-cbca86e87e3f" containerName="payload-processing"
Oct 31 21:39:17.372355 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.371698    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="387e0cb6-78f0-49f9-9a2d-395e7aeb9ee8" containerName="payload-pre-processing"
Oct 31 21:39:17.376861 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.376840    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:39:17.396802 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.396773    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-9bbfc5-openshift-default-dockercfg-sbk99\""
Oct 31 21:39:17.415220 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.415185    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"]
Oct 31 21:39:17.447722 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.447688    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:17.464464 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.464428    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istio-envoy\") pod \"e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") " pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:39:17.464666 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.464481    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-workload-certs\") pod \"e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") " pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:39:17.464666 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.464563    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istiod-ca-cert\") pod \"e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") " pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:39:17.464764 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.464671    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istio-podinfo\") pod \"e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") " pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:39:17.464764 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.464703    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istio-token\") pod \"e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") " pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:39:17.464844 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.464800    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-credential-socket\") pod \"e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") " pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:39:17.464844 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.464838    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istio-data\") pod \"e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") " pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:39:17.465028 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.465011    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-workload-socket\") pod \"e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") " pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:39:17.465076 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.465042    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kmqx\" (UniqueName: \"kubernetes.io/projected/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-kube-api-access-5kmqx\") pod \"e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") " pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:39:17.646014 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.625682    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istio-podinfo\") pod \"e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") " pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:39:17.646014 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.625736    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istio-token\") pod \"e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") " pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:39:17.646014 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.625824    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-credential-socket\") pod \"e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") " pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:39:17.646014 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.625872    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istio-data\") pod \"e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") " pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:39:17.646014 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.626009    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-workload-socket\") pod \"e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") " pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:39:17.646014 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.626041    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kmqx\" (UniqueName: \"kubernetes.io/projected/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-kube-api-access-5kmqx\") pod \"e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") " pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:39:17.646014 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.626071    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istio-envoy\") pod \"e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") " pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:39:17.646014 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.626116    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-workload-certs\") pod \"e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") " pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:39:17.646014 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.626167    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istiod-ca-cert\") pod \"e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") " pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:39:17.647052 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.647016    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istiod-ca-cert\") pod \"e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") " pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:39:17.689299 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.689259    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-credential-socket\") pod \"e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") " pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:39:17.706092 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.705385    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-workload-socket\") pod \"e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") " pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:39:17.706092 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.705678    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-workload-certs\") pod \"e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") " pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:39:17.706571 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.706549    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istio-data\") pod \"e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") " pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:39:17.727784 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.727733    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:17.806816 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.804352    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istio-envoy\") pod \"e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") " pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:39:17.812576 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.812533    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istio-podinfo\") pod \"e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") " pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:39:17.815853 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.815795    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kmqx\" (UniqueName: \"kubernetes.io/projected/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-kube-api-access-5kmqx\") pod \"e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") " pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:39:17.816369 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.816325    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istio-token\") pod \"e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") " pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:39:17.995597 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:17.995509    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:39:18.226135 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:18.226099    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="71d27085-7e15-413d-a20c-9e81d3a1123a" path="/var/lib/kubelet/pods/71d27085-7e15-413d-a20c-9e81d3a1123a/volumes"
Oct 31 21:39:18.324022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:18.323993    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"]
Oct 31 21:39:18.324336 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:39:18.324312    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd420b3f8_f5f8_4bd3_86b9_c4b6b6911041.slice/crio-59a98ce67aa2b6c8c2120ebe9e975f4cabd326468415d1d1e5e3881a903f1efb WatchSource:0}: Error finding container 59a98ce67aa2b6c8c2120ebe9e975f4cabd326468415d1d1e5e3881a903f1efb: Status 404 returned error can't find the container with id 59a98ce67aa2b6c8c2120ebe9e975f4cabd326468415d1d1e5e3881a903f1efb
Oct 31 21:39:18.327142 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:18.327112    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:39:18.327207 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:18.327178    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:39:18.327239 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:18.327208    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:39:18.627277 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:18.627187    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:18.995600 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:18.995506    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:19.135279 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:19.135231    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:19.290698 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:19.290661    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k" event={"ID":"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041","Type":"ContainerStarted","Data":"922bd049f4fd2e77e33921b23e8f2c87f6a090e9cea4e4dc185aa9a6ce105320"}
Oct 31 21:39:19.290913 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:19.290718    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k" event={"ID":"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041","Type":"ContainerStarted","Data":"59a98ce67aa2b6c8c2120ebe9e975f4cabd326468415d1d1e5e3881a903f1efb"}
Oct 31 21:39:19.353574 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:19.353522    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k" podStartSLOduration=2.353506712 podStartE2EDuration="2.353506712s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:39:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:39:19.352491873 +0000 UTC m=+1399.268801374" watchObservedRunningTime="2026-10-31 21:39:19.353506712 +0000 UTC m=+1399.269816141"
Oct 31 21:39:19.996445 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:19.996398    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:39:20.002629 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:20.002598    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:39:20.298577 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:20.298542    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:39:20.299558 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:20.299526    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:39:20.352606 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:20.352546    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:20.800053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:20.800020    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:39:21.454379 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:39:21.454339    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:39:21.777418 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:21.777381    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:21.897518 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:21.897483    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:22.498852 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:22.498812    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:23.586263 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:23.586223    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:23.703833 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:23.703797    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:23.752248 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:23.752213    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-b9bdbfb6c-kcqqn"]
Oct 31 21:39:23.756614 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:23.756592    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-b9bdbfb6c-kcqqn"
Oct 31 21:39:23.775308 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:23.775270    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-9bbfc5\""
Oct 31 21:39:23.788693 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:23.788654    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-9bbfc5-dockercfg-4nn2x\""
Oct 31 21:39:23.792637 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:23.792608    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-b9bdbfb6c-kcqqn"]
Oct 31 21:39:23.888043 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:23.883956    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f801e2e-d41d-429b-8af4-416e3f2724d3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-9bbfc5-b9bdbfb6c-kcqqn\" (UID: \"9f801e2e-d41d-429b-8af4-416e3f2724d3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-b9bdbfb6c-kcqqn"
Oct 31 21:39:23.888043 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:23.884071    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hh5kl\" (UniqueName: \"kubernetes.io/projected/9f801e2e-d41d-429b-8af4-416e3f2724d3-kube-api-access-hh5kl\") pod \"payload-pre-processing-e2e-ipp-b-9bbfc5-b9bdbfb6c-kcqqn\" (UID: \"9f801e2e-d41d-429b-8af4-416e3f2724d3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-b9bdbfb6c-kcqqn"
Oct 31 21:39:23.942964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:23.917006    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-565fd984df-pmdxb"]
Oct 31 21:39:24.038037 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:24.009110    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f801e2e-d41d-429b-8af4-416e3f2724d3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-9bbfc5-b9bdbfb6c-kcqqn\" (UID: \"9f801e2e-d41d-429b-8af4-416e3f2724d3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-b9bdbfb6c-kcqqn"
Oct 31 21:39:24.038037 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:24.009198    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hh5kl\" (UniqueName: \"kubernetes.io/projected/9f801e2e-d41d-429b-8af4-416e3f2724d3-kube-api-access-hh5kl\") pod \"payload-pre-processing-e2e-ipp-b-9bbfc5-b9bdbfb6c-kcqqn\" (UID: \"9f801e2e-d41d-429b-8af4-416e3f2724d3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-b9bdbfb6c-kcqqn"
Oct 31 21:39:24.038037 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:24.010452    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f801e2e-d41d-429b-8af4-416e3f2724d3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-9bbfc5-b9bdbfb6c-kcqqn\" (UID: \"9f801e2e-d41d-429b-8af4-416e3f2724d3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-b9bdbfb6c-kcqqn"
Oct 31 21:39:24.072959 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:24.068659    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-565fd984df-pmdxb"]
Oct 31 21:39:24.130338 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:24.127695    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hh5kl\" (UniqueName: \"kubernetes.io/projected/9f801e2e-d41d-429b-8af4-416e3f2724d3-kube-api-access-hh5kl\") pod \"payload-pre-processing-e2e-ipp-b-9bbfc5-b9bdbfb6c-kcqqn\" (UID: \"9f801e2e-d41d-429b-8af4-416e3f2724d3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-b9bdbfb6c-kcqqn"
Oct 31 21:39:24.148808 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:24.147921    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-565fd984df-pmdxb"
Oct 31 21:39:24.248453 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:24.248411    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw67d\" (UniqueName: \"kubernetes.io/projected/478a16e8-4af0-4aab-a076-903a1d3da88b-kube-api-access-tw67d\") pod \"payload-processing-e2e-ipp-b-9bbfc5-565fd984df-pmdxb\" (UID: \"478a16e8-4af0-4aab-a076-903a1d3da88b\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-565fd984df-pmdxb"
Oct 31 21:39:24.248700 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:24.248676    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/478a16e8-4af0-4aab-a076-903a1d3da88b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-9bbfc5-565fd984df-pmdxb\" (UID: \"478a16e8-4af0-4aab-a076-903a1d3da88b\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-565fd984df-pmdxb"
Oct 31 21:39:24.350755 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:24.350718    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/478a16e8-4af0-4aab-a076-903a1d3da88b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-9bbfc5-565fd984df-pmdxb\" (UID: \"478a16e8-4af0-4aab-a076-903a1d3da88b\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-565fd984df-pmdxb"
Oct 31 21:39:24.350983 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:24.350961    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw67d\" (UniqueName: \"kubernetes.io/projected/478a16e8-4af0-4aab-a076-903a1d3da88b-kube-api-access-tw67d\") pod \"payload-processing-e2e-ipp-b-9bbfc5-565fd984df-pmdxb\" (UID: \"478a16e8-4af0-4aab-a076-903a1d3da88b\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-565fd984df-pmdxb"
Oct 31 21:39:24.351648 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:24.351623    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/478a16e8-4af0-4aab-a076-903a1d3da88b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-9bbfc5-565fd984df-pmdxb\" (UID: \"478a16e8-4af0-4aab-a076-903a1d3da88b\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-565fd984df-pmdxb"
Oct 31 21:39:24.371675 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:24.371632    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-b9bdbfb6c-kcqqn"
Oct 31 21:39:24.376591 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:24.376557    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw67d\" (UniqueName: \"kubernetes.io/projected/478a16e8-4af0-4aab-a076-903a1d3da88b-kube-api-access-tw67d\") pod \"payload-processing-e2e-ipp-b-9bbfc5-565fd984df-pmdxb\" (UID: \"478a16e8-4af0-4aab-a076-903a1d3da88b\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-565fd984df-pmdxb"
Oct 31 21:39:24.461281 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:24.461125    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:24.476016 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:24.475894    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-565fd984df-pmdxb"
Oct 31 21:39:24.683065 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:24.683026    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-b9bdbfb6c-kcqqn"]
Oct 31 21:39:24.684462 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:39:24.684423    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f801e2e_d41d_429b_8af4_416e3f2724d3.slice/crio-1b3aafd5c94d9da0091a165f2e7b105b81f344b831e7e48ecd35940d226b1abc WatchSource:0}: Error finding container 1b3aafd5c94d9da0091a165f2e7b105b81f344b831e7e48ecd35940d226b1abc: Status 404 returned error can't find the container with id 1b3aafd5c94d9da0091a165f2e7b105b81f344b831e7e48ecd35940d226b1abc
Oct 31 21:39:24.881321 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:39:24.881283    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod478a16e8_4af0_4aab_a076_903a1d3da88b.slice/crio-07755278d26a7ceead3f326bc64a7c660f7b04ba5cf5786f97b7be002a4f6dc8 WatchSource:0}: Error finding container 07755278d26a7ceead3f326bc64a7c660f7b04ba5cf5786f97b7be002a4f6dc8: Status 404 returned error can't find the container with id 07755278d26a7ceead3f326bc64a7c660f7b04ba5cf5786f97b7be002a4f6dc8
Oct 31 21:39:24.881880 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:24.881854    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-565fd984df-pmdxb"]
Oct 31 21:39:25.335254 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:25.335216    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-b9bdbfb6c-kcqqn" event={"ID":"9f801e2e-d41d-429b-8af4-416e3f2724d3","Type":"ContainerStarted","Data":"81719786dd006c2a81867f93ec68d74355f3b4c2e5695c433b983e3968e6d4a9"}
Oct 31 21:39:25.335487 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:25.335279    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-b9bdbfb6c-kcqqn" event={"ID":"9f801e2e-d41d-429b-8af4-416e3f2724d3","Type":"ContainerStarted","Data":"1b3aafd5c94d9da0091a165f2e7b105b81f344b831e7e48ecd35940d226b1abc"}
Oct 31 21:39:25.335487 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:25.335301    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-b9bdbfb6c-kcqqn"
Oct 31 21:39:25.337166 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:25.337123    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-565fd984df-pmdxb" event={"ID":"478a16e8-4af0-4aab-a076-903a1d3da88b","Type":"ContainerStarted","Data":"f816e8d5c043cdaea399624adc43fc9a450601589336c1ea6efc407ec1886754"}
Oct 31 21:39:25.337327 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:25.337183    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-565fd984df-pmdxb" event={"ID":"478a16e8-4af0-4aab-a076-903a1d3da88b","Type":"ContainerStarted","Data":"07755278d26a7ceead3f326bc64a7c660f7b04ba5cf5786f97b7be002a4f6dc8"}
Oct 31 21:39:25.337327 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:25.337208    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-565fd984df-pmdxb"
Oct 31 21:39:25.380521 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:25.380471    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-b9bdbfb6c-kcqqn" podStartSLOduration=2.380457656 podStartE2EDuration="2.380457656s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:39:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:39:25.380220037 +0000 UTC m=+1405.296529466" watchObservedRunningTime="2026-10-31 21:39:25.380457656 +0000 UTC m=+1405.296767085"
Oct 31 21:39:25.425678 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:25.425621    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-565fd984df-pmdxb" podStartSLOduration=2.425606434 podStartE2EDuration="2.425606434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:39:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:39:25.423689522 +0000 UTC m=+1405.339998951" watchObservedRunningTime="2026-10-31 21:39:25.425606434 +0000 UTC m=+1405.341915969"
Oct 31 21:39:26.347856 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:26.347818    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:28.653529 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:28.653489    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:30.898863 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:30.898826    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:31.042976 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:31.035687    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:32.757121 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:32.757083    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:35.031662 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:35.031624    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:36.351057 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:36.351020    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-b9bdbfb6c-kcqqn"
Oct 31 21:39:36.354135 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:36.354107    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-565fd984df-pmdxb"
Oct 31 21:39:36.738302 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:36.738212    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:38.663773 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:38.663734    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:39.985296 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:39.985262    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:40.177234 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:40.177191    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:41.274932 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:41.274892    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:43.311206 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:43.311174    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:45.062793 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:45.062750    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:46.372050 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:46.372010    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:46.861851 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:46.861812    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:46.994609 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:46.990057    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:47.110346 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:47.110314    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:47.245533 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:47.245440    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:48.942976 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:48.942922    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:50.644700 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:50.644659    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:51.454323 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:39:51.454282    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:39:52.493758 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:52.493716    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:52.676964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:52.670073    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:54.520065 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:54.520025    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:56.342139 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:56.342102    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:58.167772 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:58.167731    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:59.913120 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:59.913081    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9"]
Oct 31 21:39:59.918079 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:59.918053    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9"
Oct 31 21:39:59.938195 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:59.938167    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9"]
Oct 31 21:39:59.945331 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:59.945296    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-fc0d65\"/\"default-dockercfg-j9jn5\""
Oct 31 21:39:59.945487 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:59.945392    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-fc0d65\"/\"ipp-route-fc0d65-kserve-self-signed-certs\""
Oct 31 21:39:59.949935 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:59.949910    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-fc0d65\"/\"openshift-service-ca.crt\""
Oct 31 21:39:59.950128 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:59.950018    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-fc0d65\"/\"kube-root-ca.crt\""
Oct 31 21:39:59.964824 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:59.964783    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:39:59.977957 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:59.977914    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e965141b-d468-4dd9-80a0-018b9af1cfc0-tmp-dir\") pod \"ipp-route-fc0d65-kserve-dcf46dc4f-b22n9\" (UID: \"e965141b-d468-4dd9-80a0-018b9af1cfc0\") " pod="ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9"
Oct 31 21:39:59.978136 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:59.978093    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnxtj\" (UniqueName: \"kubernetes.io/projected/e965141b-d468-4dd9-80a0-018b9af1cfc0-kube-api-access-hnxtj\") pod \"ipp-route-fc0d65-kserve-dcf46dc4f-b22n9\" (UID: \"e965141b-d468-4dd9-80a0-018b9af1cfc0\") " pod="ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9"
Oct 31 21:39:59.978200 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:59.978174    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e965141b-d468-4dd9-80a0-018b9af1cfc0-home\") pod \"ipp-route-fc0d65-kserve-dcf46dc4f-b22n9\" (UID: \"e965141b-d468-4dd9-80a0-018b9af1cfc0\") " pod="ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9"
Oct 31 21:39:59.978200 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:59.978194    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e965141b-d468-4dd9-80a0-018b9af1cfc0-tls-certs\") pod \"ipp-route-fc0d65-kserve-dcf46dc4f-b22n9\" (UID: \"e965141b-d468-4dd9-80a0-018b9af1cfc0\") " pod="ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9"
Oct 31 21:39:59.978351 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:59.978332    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e965141b-d468-4dd9-80a0-018b9af1cfc0-model-cache\") pod \"ipp-route-fc0d65-kserve-dcf46dc4f-b22n9\" (UID: \"e965141b-d468-4dd9-80a0-018b9af1cfc0\") " pod="ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9"
Oct 31 21:39:59.978483 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:39:59.978460    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e965141b-d468-4dd9-80a0-018b9af1cfc0-dshm\") pod \"ipp-route-fc0d65-kserve-dcf46dc4f-b22n9\" (UID: \"e965141b-d468-4dd9-80a0-018b9af1cfc0\") " pod="ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9"
Oct 31 21:40:00.080140 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:00.080096    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnxtj\" (UniqueName: \"kubernetes.io/projected/e965141b-d468-4dd9-80a0-018b9af1cfc0-kube-api-access-hnxtj\") pod \"ipp-route-fc0d65-kserve-dcf46dc4f-b22n9\" (UID: \"e965141b-d468-4dd9-80a0-018b9af1cfc0\") " pod="ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9"
Oct 31 21:40:00.080344 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:00.080169    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e965141b-d468-4dd9-80a0-018b9af1cfc0-home\") pod \"ipp-route-fc0d65-kserve-dcf46dc4f-b22n9\" (UID: \"e965141b-d468-4dd9-80a0-018b9af1cfc0\") " pod="ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9"
Oct 31 21:40:00.080344 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:00.080186    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e965141b-d468-4dd9-80a0-018b9af1cfc0-tls-certs\") pod \"ipp-route-fc0d65-kserve-dcf46dc4f-b22n9\" (UID: \"e965141b-d468-4dd9-80a0-018b9af1cfc0\") " pod="ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9"
Oct 31 21:40:00.080344 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:00.080296    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e965141b-d468-4dd9-80a0-018b9af1cfc0-model-cache\") pod \"ipp-route-fc0d65-kserve-dcf46dc4f-b22n9\" (UID: \"e965141b-d468-4dd9-80a0-018b9af1cfc0\") " pod="ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9"
Oct 31 21:40:00.080502 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:00.080395    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e965141b-d468-4dd9-80a0-018b9af1cfc0-dshm\") pod \"ipp-route-fc0d65-kserve-dcf46dc4f-b22n9\" (UID: \"e965141b-d468-4dd9-80a0-018b9af1cfc0\") " pod="ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9"
Oct 31 21:40:00.080559 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:00.080547    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e965141b-d468-4dd9-80a0-018b9af1cfc0-tmp-dir\") pod \"ipp-route-fc0d65-kserve-dcf46dc4f-b22n9\" (UID: \"e965141b-d468-4dd9-80a0-018b9af1cfc0\") " pod="ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9"
Oct 31 21:40:00.080660 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:00.080634    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e965141b-d468-4dd9-80a0-018b9af1cfc0-home\") pod \"ipp-route-fc0d65-kserve-dcf46dc4f-b22n9\" (UID: \"e965141b-d468-4dd9-80a0-018b9af1cfc0\") " pod="ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9"
Oct 31 21:40:00.080725 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:00.080668    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e965141b-d468-4dd9-80a0-018b9af1cfc0-model-cache\") pod \"ipp-route-fc0d65-kserve-dcf46dc4f-b22n9\" (UID: \"e965141b-d468-4dd9-80a0-018b9af1cfc0\") " pod="ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9"
Oct 31 21:40:00.080824 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:00.080804    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e965141b-d468-4dd9-80a0-018b9af1cfc0-tmp-dir\") pod \"ipp-route-fc0d65-kserve-dcf46dc4f-b22n9\" (UID: \"e965141b-d468-4dd9-80a0-018b9af1cfc0\") " pod="ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9"
Oct 31 21:40:00.082727 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:00.082700    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e965141b-d468-4dd9-80a0-018b9af1cfc0-dshm\") pod \"ipp-route-fc0d65-kserve-dcf46dc4f-b22n9\" (UID: \"e965141b-d468-4dd9-80a0-018b9af1cfc0\") " pod="ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9"
Oct 31 21:40:00.082834 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:00.082823    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e965141b-d468-4dd9-80a0-018b9af1cfc0-tls-certs\") pod \"ipp-route-fc0d65-kserve-dcf46dc4f-b22n9\" (UID: \"e965141b-d468-4dd9-80a0-018b9af1cfc0\") " pod="ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9"
Oct 31 21:40:00.105723 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:00.105693    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnxtj\" (UniqueName: \"kubernetes.io/projected/e965141b-d468-4dd9-80a0-018b9af1cfc0-kube-api-access-hnxtj\") pod \"ipp-route-fc0d65-kserve-dcf46dc4f-b22n9\" (UID: \"e965141b-d468-4dd9-80a0-018b9af1cfc0\") " pod="ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9"
Oct 31 21:40:00.338279 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:00.334900    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-fc0d65\"/\"default-dockercfg-j9jn5\""
Oct 31 21:40:00.338617 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:00.338586    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9"
Oct 31 21:40:00.579960 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:00.577595    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:00.697696 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:00.697662    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9"]
Oct 31 21:40:00.698390 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:40:00.698365    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode965141b_d468_4dd9_80a0_018b9af1cfc0.slice/crio-2a55348fe85dd08dc2f71a41520c9247ca45146c5da218ee09fe93942c583063 WatchSource:0}: Error finding container 2a55348fe85dd08dc2f71a41520c9247ca45146c5da218ee09fe93942c583063: Status 404 returned error can't find the container with id 2a55348fe85dd08dc2f71a41520c9247ca45146c5da218ee09fe93942c583063
Oct 31 21:40:01.050964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:01.045588    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:40:01.714073 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:01.714020    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9" event={"ID":"e965141b-d468-4dd9-80a0-018b9af1cfc0","Type":"ContainerStarted","Data":"dc87eafd3a96beb66e31cef528eaeb09720556834e2b54d8019c46cd483a1e86"}
Oct 31 21:40:01.714266 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:01.714086    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9" event={"ID":"e965141b-d468-4dd9-80a0-018b9af1cfc0","Type":"ContainerStarted","Data":"2a55348fe85dd08dc2f71a41520c9247ca45146c5da218ee09fe93942c583063"}
Oct 31 21:40:01.763038 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:01.762978    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9" podStartSLOduration=2.762963132 podStartE2EDuration="2.762963132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:39:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:40:01.759288874 +0000 UTC m=+1441.675598304" watchObservedRunningTime="2026-10-31 21:40:01.762963132 +0000 UTC m=+1441.679272561"
Oct 31 21:40:02.344020 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:02.341184    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9"
Oct 31 21:40:02.825189 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:02.825147    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:04.785510 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:04.785473    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:06.722413 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:06.722375    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:07.244197 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:07.244162    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:08.251361 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:08.251328    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:09.009145 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:09.009108    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:10.339233 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:10.339193    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9"
Oct 31 21:40:10.351801 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:10.351771    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9"
Oct 31 21:40:10.791692 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:10.791660    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9"
Oct 31 21:40:11.071113 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:11.071011    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:13.329109 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:13.329072    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:15.108196 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:15.108159    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:16.464854 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:16.464817    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:16.640968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:16.638182    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:17.069046 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:17.069000    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-752623-6bc5bf56b6-pm4z4"]
Oct 31 21:40:17.233963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:17.227578    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-752623-6bc5bf56b6-pm4z4"]
Oct 31 21:40:17.237167 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:17.234077    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-752623-6bc5bf56b6-pm4z4"
Oct 31 21:40:17.269478 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:17.269335    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-752623-dockercfg-4fd56\""
Oct 31 21:40:17.269478 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:17.269435    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-752623\""
Oct 31 21:40:17.282921 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:17.282882    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-752623-6bc5bf56b6-pm4z4\" (UID: \"c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-752623-6bc5bf56b6-pm4z4"
Oct 31 21:40:17.283153 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:17.283130    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6r4t\" (UniqueName: \"kubernetes.io/projected/c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356-kube-api-access-g6r4t\") pod \"payload-processing-e2e-ipp-b-752623-6bc5bf56b6-pm4z4\" (UID: \"c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-752623-6bc5bf56b6-pm4z4"
Oct 31 21:40:17.385202 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:17.385117    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-752623-6bc5bf56b6-pm4z4\" (UID: \"c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-752623-6bc5bf56b6-pm4z4"
Oct 31 21:40:17.385365 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:17.385244    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6r4t\" (UniqueName: \"kubernetes.io/projected/c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356-kube-api-access-g6r4t\") pod \"payload-processing-e2e-ipp-b-752623-6bc5bf56b6-pm4z4\" (UID: \"c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-752623-6bc5bf56b6-pm4z4"
Oct 31 21:40:17.385820 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:17.385799    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-752623-6bc5bf56b6-pm4z4\" (UID: \"c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-752623-6bc5bf56b6-pm4z4"
Oct 31 21:40:17.412019 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:17.411969    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6r4t\" (UniqueName: \"kubernetes.io/projected/c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356-kube-api-access-g6r4t\") pod \"payload-processing-e2e-ipp-b-752623-6bc5bf56b6-pm4z4\" (UID: \"c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-752623-6bc5bf56b6-pm4z4"
Oct 31 21:40:17.565129 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:17.565094    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-752623-6bc5bf56b6-pm4z4"
Oct 31 21:40:17.738757 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:17.738665    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:17.895835 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:17.895805    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-752623-6bc5bf56b6-pm4z4"]
Oct 31 21:40:17.896773 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:40:17.896743    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0c0a5d2_db98_4e2a_a7eb_61aabb8fa356.slice/crio-5a925e3089a8709901072dc35915fda8257603c203bcf243e66891f2762e24ad WatchSource:0}: Error finding container 5a925e3089a8709901072dc35915fda8257603c203bcf243e66891f2762e24ad: Status 404 returned error can't find the container with id 5a925e3089a8709901072dc35915fda8257603c203bcf243e66891f2762e24ad
Oct 31 21:40:17.898860 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:17.898841    4460 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 21:40:18.890330 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:18.890289    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-752623-6bc5bf56b6-pm4z4" event={"ID":"c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356","Type":"ContainerStarted","Data":"5a925e3089a8709901072dc35915fda8257603c203bcf243e66891f2762e24ad"}
Oct 31 21:40:19.612326 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:19.612296    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:19.899089 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:19.898981    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-752623-6bc5bf56b6-pm4z4" event={"ID":"c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356","Type":"ContainerStarted","Data":"602b7206297a2f8f101f84a62d827f8e1890aaae6d6713893f61dd7660f47573"}
Oct 31 21:40:19.899089 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:19.899068    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-752623-6bc5bf56b6-pm4z4"
Oct 31 21:40:19.946769 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:19.946711    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-752623-6bc5bf56b6-pm4z4" podStartSLOduration=2.241826464 podStartE2EDuration="3.946695933s" totalImagesPullingTime="1.704869469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:40:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:40:19.943252839 +0000 UTC m=+1459.859562268" watchObservedRunningTime="2026-10-31 21:40:19.946695933 +0000 UTC m=+1459.863005363"
Oct 31 21:40:20.303960 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:20.303307    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:20.460970 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:20.450996    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:21.335959 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:21.335406    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:21.501368 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:40:21.501332    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:40:22.361965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:22.358606    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:22.478402 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:22.478358    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:23.040869 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:23.040835    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:24.836978 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:24.834016    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:26.704907 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:26.704862    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:28.054488 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:28.054451    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-565fd984df-pmdxb"]
Oct 31 21:40:28.088580 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:28.088546    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-b9bdbfb6c-kcqqn"]
Oct 31 21:40:28.210961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:28.210898    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:28.350283 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:28.350155    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:28.461905 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:28.461865    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:28.588733 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:28.588676    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:30.976966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:30.972000    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-752623-6bc5bf56b6-pm4z4"
Oct 31 21:40:32.122292 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:32.122251    4460 generic.go:356] "Generic (PLEG): container finished" podID="0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9" containerID="ec253cef5e4639cf518d7b9e22cfb51844f7860cee985b4d87e91546777fba30" exitCode=0
Oct 31 21:40:32.122756 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:32.122320    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td" event={"ID":"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9","Type":"ContainerDied","Data":"ec253cef5e4639cf518d7b9e22cfb51844f7860cee985b4d87e91546777fba30"}
Oct 31 21:40:32.122822 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:32.122794    4460 scope.go:122] "RemoveContainer" containerID="ec253cef5e4639cf518d7b9e22cfb51844f7860cee985b4d87e91546777fba30"
Oct 31 21:40:32.124610 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:32.124582    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:40:32.124688 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:32.124653    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:40:32.124688 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:32.124683    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:40:32.232042 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:32.231994    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:33.009632 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:33.009595    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:33.061547 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:33.061502    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-565fd984df-pmdxb" podUID="478a16e8-4af0-4aab-a076-903a1d3da88b" containerName="payload-processing" containerID="cri-o://f816e8d5c043cdaea399624adc43fc9a450601589336c1ea6efc407ec1886754" gracePeriod=25
Oct 31 21:40:33.094331 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:33.094284    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-b9bdbfb6c-kcqqn" podUID="9f801e2e-d41d-429b-8af4-416e3f2724d3" containerName="payload-pre-processing" containerID="cri-o://81719786dd006c2a81867f93ec68d74355f3b4c2e5695c433b983e3968e6d4a9" gracePeriod=25
Oct 31 21:40:33.140962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:33.140802    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td" event={"ID":"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9","Type":"ContainerStarted","Data":"4b3a8d0019acff71fa51bf90cc1b41c39d89266cf70bc5e29f7549c501fb5c73"}
Oct 31 21:40:33.431559 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:33.431126    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-565fd984df-pmdxb"
Oct 31 21:40:33.513417 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:33.512213    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/478a16e8-4af0-4aab-a076-903a1d3da88b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/478a16e8-4af0-4aab-a076-903a1d3da88b-plugins-config-volume\") pod \"478a16e8-4af0-4aab-a076-903a1d3da88b\" (UID: \"478a16e8-4af0-4aab-a076-903a1d3da88b\") "
Oct 31 21:40:33.513417 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:33.512284    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/478a16e8-4af0-4aab-a076-903a1d3da88b-kube-api-access-tw67d\" (UniqueName: \"kubernetes.io/projected/478a16e8-4af0-4aab-a076-903a1d3da88b-kube-api-access-tw67d\") pod \"478a16e8-4af0-4aab-a076-903a1d3da88b\" (UID: \"478a16e8-4af0-4aab-a076-903a1d3da88b\") "
Oct 31 21:40:33.513722 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:33.513695    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/478a16e8-4af0-4aab-a076-903a1d3da88b-plugins-config-volume" pod "478a16e8-4af0-4aab-a076-903a1d3da88b" (UID: "478a16e8-4af0-4aab-a076-903a1d3da88b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:40:33.516040 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:33.516011    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/478a16e8-4af0-4aab-a076-903a1d3da88b-kube-api-access-tw67d" pod "478a16e8-4af0-4aab-a076-903a1d3da88b" (UID: "478a16e8-4af0-4aab-a076-903a1d3da88b"). InnerVolumeSpecName "kube-api-access-tw67d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:40:33.574821 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:33.574789    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-b9bdbfb6c-kcqqn"
Oct 31 21:40:33.614507 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:33.614474    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9f801e2e-d41d-429b-8af4-416e3f2724d3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f801e2e-d41d-429b-8af4-416e3f2724d3-plugins-config-volume\") pod \"9f801e2e-d41d-429b-8af4-416e3f2724d3\" (UID: \"9f801e2e-d41d-429b-8af4-416e3f2724d3\") "
Oct 31 21:40:33.614657 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:33.614575    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9f801e2e-d41d-429b-8af4-416e3f2724d3-kube-api-access-hh5kl\" (UniqueName: \"kubernetes.io/projected/9f801e2e-d41d-429b-8af4-416e3f2724d3-kube-api-access-hh5kl\") pod \"9f801e2e-d41d-429b-8af4-416e3f2724d3\" (UID: \"9f801e2e-d41d-429b-8af4-416e3f2724d3\") "
Oct 31 21:40:33.614924 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:33.614896    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f801e2e-d41d-429b-8af4-416e3f2724d3-plugins-config-volume" pod "9f801e2e-d41d-429b-8af4-416e3f2724d3" (UID: "9f801e2e-d41d-429b-8af4-416e3f2724d3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:40:33.615052 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:33.615036    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f801e2e-d41d-429b-8af4-416e3f2724d3-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:40:33.615112 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:33.615054    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/478a16e8-4af0-4aab-a076-903a1d3da88b-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:40:33.615112 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:33.615064    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tw67d\" (UniqueName: \"kubernetes.io/projected/478a16e8-4af0-4aab-a076-903a1d3da88b-kube-api-access-tw67d\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:40:33.616756 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:33.616726    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f801e2e-d41d-429b-8af4-416e3f2724d3-kube-api-access-hh5kl" pod "9f801e2e-d41d-429b-8af4-416e3f2724d3" (UID: "9f801e2e-d41d-429b-8af4-416e3f2724d3"). InnerVolumeSpecName "kube-api-access-hh5kl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:40:33.716738 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:33.716645    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hh5kl\" (UniqueName: \"kubernetes.io/projected/9f801e2e-d41d-429b-8af4-416e3f2724d3-kube-api-access-hh5kl\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:40:33.819868 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:33.819785    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:40:33.827554 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:33.827079    4460 patch_prober.go:29] interesting pod/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused" start-of-body=
Oct 31 21:40:33.827554 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:33.827185    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td" podUID="0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused"
Oct 31 21:40:34.153449 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:34.153413    4460 generic.go:356] "Generic (PLEG): container finished" podID="d420b3f8-f5f8-4bd3-86b9-c4b6b6911041" containerID="922bd049f4fd2e77e33921b23e8f2c87f6a090e9cea4e4dc185aa9a6ce105320" exitCode=0
Oct 31 21:40:34.153899 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:34.153492    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k" event={"ID":"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041","Type":"ContainerDied","Data":"922bd049f4fd2e77e33921b23e8f2c87f6a090e9cea4e4dc185aa9a6ce105320"}
Oct 31 21:40:34.154093 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:34.154071    4460 scope.go:122] "RemoveContainer" containerID="922bd049f4fd2e77e33921b23e8f2c87f6a090e9cea4e4dc185aa9a6ce105320"
Oct 31 21:40:34.155034 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:34.155011    4460 generic.go:356] "Generic (PLEG): container finished" podID="9f801e2e-d41d-429b-8af4-416e3f2724d3" containerID="81719786dd006c2a81867f93ec68d74355f3b4c2e5695c433b983e3968e6d4a9" exitCode=0
Oct 31 21:40:34.155118 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:34.155072    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-b9bdbfb6c-kcqqn"
Oct 31 21:40:34.155118 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:34.155099    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-b9bdbfb6c-kcqqn" event={"ID":"9f801e2e-d41d-429b-8af4-416e3f2724d3","Type":"ContainerDied","Data":"81719786dd006c2a81867f93ec68d74355f3b4c2e5695c433b983e3968e6d4a9"}
Oct 31 21:40:34.155226 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:34.155157    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-b9bdbfb6c-kcqqn" event={"ID":"9f801e2e-d41d-429b-8af4-416e3f2724d3","Type":"ContainerDied","Data":"1b3aafd5c94d9da0091a165f2e7b105b81f344b831e7e48ecd35940d226b1abc"}
Oct 31 21:40:34.155226 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:34.155187    4460 scope.go:122] "RemoveContainer" containerID="81719786dd006c2a81867f93ec68d74355f3b4c2e5695c433b983e3968e6d4a9"
Oct 31 21:40:34.155918 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:34.155889    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:40:34.156034 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:34.155995    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:40:34.156104 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:34.156047    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:40:34.157141 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:34.157117    4460 generic.go:356] "Generic (PLEG): container finished" podID="478a16e8-4af0-4aab-a076-903a1d3da88b" containerID="f816e8d5c043cdaea399624adc43fc9a450601589336c1ea6efc407ec1886754" exitCode=0
Oct 31 21:40:34.157478 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:34.157464    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-565fd984df-pmdxb"
Oct 31 21:40:34.157634 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:34.157610    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-565fd984df-pmdxb" event={"ID":"478a16e8-4af0-4aab-a076-903a1d3da88b","Type":"ContainerDied","Data":"f816e8d5c043cdaea399624adc43fc9a450601589336c1ea6efc407ec1886754"}
Oct 31 21:40:34.157721 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:34.157658    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-565fd984df-pmdxb" event={"ID":"478a16e8-4af0-4aab-a076-903a1d3da88b","Type":"ContainerDied","Data":"07755278d26a7ceead3f326bc64a7c660f7b04ba5cf5786f97b7be002a4f6dc8"}
Oct 31 21:40:34.171791 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:34.171770    4460 scope.go:122] "RemoveContainer" containerID="81719786dd006c2a81867f93ec68d74355f3b4c2e5695c433b983e3968e6d4a9"
Oct 31 21:40:34.172164 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:40:34.172142    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81719786dd006c2a81867f93ec68d74355f3b4c2e5695c433b983e3968e6d4a9\": container with ID starting with 81719786dd006c2a81867f93ec68d74355f3b4c2e5695c433b983e3968e6d4a9 not found: ID does not exist" containerID="81719786dd006c2a81867f93ec68d74355f3b4c2e5695c433b983e3968e6d4a9"
Oct 31 21:40:34.172229 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:34.172174    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81719786dd006c2a81867f93ec68d74355f3b4c2e5695c433b983e3968e6d4a9"} err="failed to get container status \"81719786dd006c2a81867f93ec68d74355f3b4c2e5695c433b983e3968e6d4a9\": rpc error: code = NotFound desc = could not find container \"81719786dd006c2a81867f93ec68d74355f3b4c2e5695c433b983e3968e6d4a9\": container with ID starting with 81719786dd006c2a81867f93ec68d74355f3b4c2e5695c433b983e3968e6d4a9 not found: ID does not exist"
Oct 31 21:40:34.172229 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:34.172194    4460 scope.go:122] "RemoveContainer" containerID="f816e8d5c043cdaea399624adc43fc9a450601589336c1ea6efc407ec1886754"
Oct 31 21:40:34.207982 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:34.207836    4460 scope.go:122] "RemoveContainer" containerID="f816e8d5c043cdaea399624adc43fc9a450601589336c1ea6efc407ec1886754"
Oct 31 21:40:34.208467 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:40:34.208435    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f816e8d5c043cdaea399624adc43fc9a450601589336c1ea6efc407ec1886754\": container with ID starting with f816e8d5c043cdaea399624adc43fc9a450601589336c1ea6efc407ec1886754 not found: ID does not exist" containerID="f816e8d5c043cdaea399624adc43fc9a450601589336c1ea6efc407ec1886754"
Oct 31 21:40:34.208578 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:34.208476    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f816e8d5c043cdaea399624adc43fc9a450601589336c1ea6efc407ec1886754"} err="failed to get container status \"f816e8d5c043cdaea399624adc43fc9a450601589336c1ea6efc407ec1886754\": rpc error: code = NotFound desc = could not find container \"f816e8d5c043cdaea399624adc43fc9a450601589336c1ea6efc407ec1886754\": container with ID starting with f816e8d5c043cdaea399624adc43fc9a450601589336c1ea6efc407ec1886754 not found: ID does not exist"
Oct 31 21:40:34.261983 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:34.259957    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-b9bdbfb6c-kcqqn"]
Oct 31 21:40:34.263310 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:34.263275    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-b9bdbfb6c-kcqqn"]
Oct 31 21:40:34.293281 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:34.293214    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-565fd984df-pmdxb"]
Oct 31 21:40:34.298356 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:34.298305    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-565fd984df-pmdxb"]
Oct 31 21:40:34.797261 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:34.794302    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:34.845964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:34.845401    4460 patch_prober.go:29] interesting pod/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused" start-of-body=
Oct 31 21:40:34.845964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:34.845470    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td" podUID="0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused"
Oct 31 21:40:35.178287 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:35.178196    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k" event={"ID":"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041","Type":"ContainerStarted","Data":"b5396875d149dbf3ccc4a5d3e536abc90adea3c634dd53f17908910da3a26364"}
Oct 31 21:40:35.262173 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:35.262126    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:40:35.298794 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:35.298751    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:40:35.834964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:35.820365    4460 patch_prober.go:29] interesting pod/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused" start-of-body=
Oct 31 21:40:35.835326 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:35.835278    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td" podUID="0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused"
Oct 31 21:40:35.995793 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:35.995759    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:40:35.997006 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:35.996979    4460 patch_prober.go:29] interesting pod/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused" start-of-body=
Oct 31 21:40:35.997073 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:35.997042    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k" podUID="d420b3f8-f5f8-4bd3-86b9-c4b6b6911041" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused"
Oct 31 21:40:36.269993 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:36.264833    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="478a16e8-4af0-4aab-a076-903a1d3da88b" path="/var/lib/kubelet/pods/478a16e8-4af0-4aab-a076-903a1d3da88b/volumes"
Oct 31 21:40:36.269993 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:36.265627    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9f801e2e-d41d-429b-8af4-416e3f2724d3" path="/var/lib/kubelet/pods/9f801e2e-d41d-429b-8af4-416e3f2724d3/volumes"
Oct 31 21:40:36.837974 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:36.830067    4460 patch_prober.go:29] interesting pod/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused" start-of-body=
Oct 31 21:40:36.837974 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:36.830143    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td" podUID="0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused"
Oct 31 21:40:36.996346 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:36.996303    4460 patch_prober.go:29] interesting pod/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused" start-of-body=
Oct 31 21:40:36.996503 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:36.996359    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k" podUID="d420b3f8-f5f8-4bd3-86b9-c4b6b6911041" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused"
Oct 31 21:40:37.190658 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:37.190577    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:37.834965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:37.824364    4460 patch_prober.go:29] interesting pod/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused" start-of-body=
Oct 31 21:40:37.834965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:37.824435    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td" podUID="0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.108:15021/healthz/ready\": dial tcp 10.133.0.108:15021: connect: connection refused"
Oct 31 21:40:37.996620 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:37.996584    4460 patch_prober.go:29] interesting pod/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused" start-of-body=
Oct 31 21:40:37.996858 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:37.996831    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k" podUID="d420b3f8-f5f8-4bd3-86b9-c4b6b6911041" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused"
Oct 31 21:40:38.845463 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:38.845427    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:40:38.847811 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:38.847786    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:40:38.912303 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:38.912268    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:39.017662 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:39.017617    4460 patch_prober.go:29] interesting pod/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused" start-of-body=
Oct 31 21:40:39.018485 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:39.018448    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k" podUID="d420b3f8-f5f8-4bd3-86b9-c4b6b6911041" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused"
Oct 31 21:40:39.996145 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:39.996107    4460 patch_prober.go:29] interesting pod/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused" start-of-body=
Oct 31 21:40:39.996543 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:39.996162    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k" podUID="d420b3f8-f5f8-4bd3-86b9-c4b6b6911041" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused"
Oct 31 21:40:40.624746 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:40.624707    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:41.001519 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:41.001434    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:40:41.002618 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:41.002592    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:40:42.817156 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:42.817116    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:44.483416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:44.483374    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:45.083983 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:45.083923    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:40:46.487884 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:46.487851    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:46.650238 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:46.650201    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:48.303162 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:48.303120    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:50.020188 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:50.020151    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:51.454017 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:40:51.453977    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:40:51.740798 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:51.740709    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:53.473088 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:53.472995    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:54.669403 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:54.669371    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:54.897620 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:54.897559    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:55.078805 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:55.078754    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-6d947544ff-6kxwd"]
Oct 31 21:40:55.079822 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:55.079799    4460 cpu_manager.go:414] "containerMap: removing container" podUID="478a16e8-4af0-4aab-a076-903a1d3da88b" containerName="payload-processing"
Oct 31 21:40:55.079883 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:55.079828    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="478a16e8-4af0-4aab-a076-903a1d3da88b" containerName="payload-processing"
Oct 31 21:40:55.079920 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:55.079900    4460 cpu_manager.go:414] "containerMap: removing container" podUID="9f801e2e-d41d-429b-8af4-416e3f2724d3" containerName="payload-pre-processing"
Oct 31 21:40:55.079920 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:55.079909    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9f801e2e-d41d-429b-8af4-416e3f2724d3" containerName="payload-pre-processing"
Oct 31 21:40:55.080110 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:55.080096    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="9f801e2e-d41d-429b-8af4-416e3f2724d3" containerName="payload-pre-processing"
Oct 31 21:40:55.080147 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:55.080122    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="478a16e8-4af0-4aab-a076-903a1d3da88b" containerName="payload-processing"
Oct 31 21:40:55.084546 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:55.084521    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-6d947544ff-6kxwd"
Oct 31 21:40:55.101536 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:55.101501    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-6d947544ff-6kxwd"]
Oct 31 21:40:55.111377 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:55.111343    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-9bbfc5-dockercfg-m7jjr\""
Oct 31 21:40:55.125212 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:55.125175    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-9bbfc5\""
Oct 31 21:40:55.391134 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:55.357183    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/180d2d9e-b101-45fc-a814-368a8efab6f7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-9bbfc5-6d947544ff-6kxwd\" (UID: \"180d2d9e-b101-45fc-a814-368a8efab6f7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-6d947544ff-6kxwd"
Oct 31 21:40:55.391134 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:55.357408    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kzqb\" (UniqueName: \"kubernetes.io/projected/180d2d9e-b101-45fc-a814-368a8efab6f7-kube-api-access-4kzqb\") pod \"payload-pre-processing-e2e-ipp-b-9bbfc5-6d947544ff-6kxwd\" (UID: \"180d2d9e-b101-45fc-a814-368a8efab6f7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-6d947544ff-6kxwd"
Oct 31 21:40:55.401915 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:55.401715    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-7b68fff8cd-t85fc"]
Oct 31 21:40:55.429962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:55.417559    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-7b68fff8cd-t85fc"]
Oct 31 21:40:55.429962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:55.417686    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-7b68fff8cd-t85fc"
Oct 31 21:40:55.461986 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:55.460461    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/180d2d9e-b101-45fc-a814-368a8efab6f7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-9bbfc5-6d947544ff-6kxwd\" (UID: \"180d2d9e-b101-45fc-a814-368a8efab6f7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-6d947544ff-6kxwd"
Oct 31 21:40:55.461986 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:55.460741    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kzqb\" (UniqueName: \"kubernetes.io/projected/180d2d9e-b101-45fc-a814-368a8efab6f7-kube-api-access-4kzqb\") pod \"payload-pre-processing-e2e-ipp-b-9bbfc5-6d947544ff-6kxwd\" (UID: \"180d2d9e-b101-45fc-a814-368a8efab6f7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-6d947544ff-6kxwd"
Oct 31 21:40:55.467986 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:55.462494    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/180d2d9e-b101-45fc-a814-368a8efab6f7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-9bbfc5-6d947544ff-6kxwd\" (UID: \"180d2d9e-b101-45fc-a814-368a8efab6f7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-6d947544ff-6kxwd"
Oct 31 21:40:55.498021 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:55.497387    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kzqb\" (UniqueName: \"kubernetes.io/projected/180d2d9e-b101-45fc-a814-368a8efab6f7-kube-api-access-4kzqb\") pod \"payload-pre-processing-e2e-ipp-b-9bbfc5-6d947544ff-6kxwd\" (UID: \"180d2d9e-b101-45fc-a814-368a8efab6f7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-6d947544ff-6kxwd"
Oct 31 21:40:55.556267 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:55.556225    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:55.562678 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:55.562646    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpkgg\" (UniqueName: \"kubernetes.io/projected/035ef563-8474-46f9-9f37-0284c7f4721e-kube-api-access-qpkgg\") pod \"payload-processing-e2e-ipp-b-9bbfc5-7b68fff8cd-t85fc\" (UID: \"035ef563-8474-46f9-9f37-0284c7f4721e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-7b68fff8cd-t85fc"
Oct 31 21:40:55.563143 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:55.563114    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/035ef563-8474-46f9-9f37-0284c7f4721e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-9bbfc5-7b68fff8cd-t85fc\" (UID: \"035ef563-8474-46f9-9f37-0284c7f4721e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-7b68fff8cd-t85fc"
Oct 31 21:40:55.664429 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:55.664340    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/035ef563-8474-46f9-9f37-0284c7f4721e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-9bbfc5-7b68fff8cd-t85fc\" (UID: \"035ef563-8474-46f9-9f37-0284c7f4721e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-7b68fff8cd-t85fc"
Oct 31 21:40:55.664605 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:55.664464    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpkgg\" (UniqueName: \"kubernetes.io/projected/035ef563-8474-46f9-9f37-0284c7f4721e-kube-api-access-qpkgg\") pod \"payload-processing-e2e-ipp-b-9bbfc5-7b68fff8cd-t85fc\" (UID: \"035ef563-8474-46f9-9f37-0284c7f4721e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-7b68fff8cd-t85fc"
Oct 31 21:40:55.665203 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:55.665176    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/035ef563-8474-46f9-9f37-0284c7f4721e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-9bbfc5-7b68fff8cd-t85fc\" (UID: \"035ef563-8474-46f9-9f37-0284c7f4721e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-7b68fff8cd-t85fc"
Oct 31 21:40:55.691119 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:55.691076    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpkgg\" (UniqueName: \"kubernetes.io/projected/035ef563-8474-46f9-9f37-0284c7f4721e-kube-api-access-qpkgg\") pod \"payload-processing-e2e-ipp-b-9bbfc5-7b68fff8cd-t85fc\" (UID: \"035ef563-8474-46f9-9f37-0284c7f4721e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-7b68fff8cd-t85fc"
Oct 31 21:40:55.709254 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:55.709223    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-6d947544ff-6kxwd"
Oct 31 21:40:55.788857 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:55.786283    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-7b68fff8cd-t85fc"
Oct 31 21:40:56.189961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:56.189906    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:56.309995 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:56.308676    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:56.425232 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:56.424897    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:56.505170 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:56.505139    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-6d947544ff-6kxwd"]
Oct 31 21:40:56.507383 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:40:56.507349    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod180d2d9e_b101_45fc_a814_368a8efab6f7.slice/crio-29fc1561bdb1d2f6efb31815eb0e69156b0011c02b72d336bc8da99d0e446c99 WatchSource:0}: Error finding container 29fc1561bdb1d2f6efb31815eb0e69156b0011c02b72d336bc8da99d0e446c99: Status 404 returned error can't find the container with id 29fc1561bdb1d2f6efb31815eb0e69156b0011c02b72d336bc8da99d0e446c99
Oct 31 21:40:56.529432 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:56.529400    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-7b68fff8cd-t85fc"]
Oct 31 21:40:56.530312 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:40:56.530271    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod035ef563_8474_46f9_9f37_0284c7f4721e.slice/crio-5eb365629feb5d57b68e9a53bc2807047dc9ce906fb5656b1992d6d90addba46 WatchSource:0}: Error finding container 5eb365629feb5d57b68e9a53bc2807047dc9ce906fb5656b1992d6d90addba46: Status 404 returned error can't find the container with id 5eb365629feb5d57b68e9a53bc2807047dc9ce906fb5656b1992d6d90addba46
Oct 31 21:40:56.869150 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:56.869112    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:40:57.482928 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:57.482883    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-7b68fff8cd-t85fc" event={"ID":"035ef563-8474-46f9-9f37-0284c7f4721e","Type":"ContainerStarted","Data":"6717754983b8c4daf7226d2f822cf2890c831e12a2966d6d1fc6c5bcf47c6945"}
Oct 31 21:40:57.483491 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:57.482979    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-7b68fff8cd-t85fc"
Oct 31 21:40:57.483491 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:57.483004    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-7b68fff8cd-t85fc" event={"ID":"035ef563-8474-46f9-9f37-0284c7f4721e","Type":"ContainerStarted","Data":"5eb365629feb5d57b68e9a53bc2807047dc9ce906fb5656b1992d6d90addba46"}
Oct 31 21:40:57.484674 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:57.484640    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-6d947544ff-6kxwd" event={"ID":"180d2d9e-b101-45fc-a814-368a8efab6f7","Type":"ContainerStarted","Data":"bb3526e6aa28c48baf28496123b22e53cf9db5d72651f7855eccd7dd2ae99b07"}
Oct 31 21:40:57.484780 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:57.484703    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-6d947544ff-6kxwd" event={"ID":"180d2d9e-b101-45fc-a814-368a8efab6f7","Type":"ContainerStarted","Data":"29fc1561bdb1d2f6efb31815eb0e69156b0011c02b72d336bc8da99d0e446c99"}
Oct 31 21:40:57.484780 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:57.484731    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-6d947544ff-6kxwd"
Oct 31 21:40:57.530654 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:57.530598    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-7b68fff8cd-t85fc" podStartSLOduration=2.530584308 podStartE2EDuration="2.530584308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:40:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:40:57.527293837 +0000 UTC m=+1497.443603289" watchObservedRunningTime="2026-10-31 21:40:57.530584308 +0000 UTC m=+1497.446893738"
Oct 31 21:40:57.575189 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:57.575135    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-6d947544ff-6kxwd" podStartSLOduration=2.575119577 podStartE2EDuration="2.575119577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:40:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:40:57.573671454 +0000 UTC m=+1497.489980886" watchObservedRunningTime="2026-10-31 21:40:57.575119577 +0000 UTC m=+1497.491429069"
Oct 31 21:40:58.136018 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:40:58.135984    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:00.341678 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:00.341636    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:00.511861 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:00.511834    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ssl9_26069598-5dc4-4c96-b55f-dd2f09884419/ovn-acl-logging/0.log"
Oct 31 21:41:00.514320 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:00.514284    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ssl9_26069598-5dc4-4c96-b55f-dd2f09884419/ovn-acl-logging/0.log"
Oct 31 21:41:02.637478 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:02.637419    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:03.622503 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:03.622458    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:03.742038 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:03.742003    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:03.929822 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:03.929716    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"]
Oct 31 21:41:03.930084 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:03.930052    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td" podUID="0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9" containerName="istio-proxy" containerID="cri-o://4b3a8d0019acff71fa51bf90cc1b41c39d89266cf70bc5e29f7549c501fb5c73" gracePeriod=30
Oct 31 21:41:03.952788 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:03.952741    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:04.001865 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:04.001832    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2"]
Oct 31 21:41:04.002138 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:04.002110    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2" podUID="e8fc85a9-ec28-4322-a395-cc68f6a65947" containerName="maas-api" containerID="cri-o://f9fa14423d5896c7ce20f4d4857396fa8f079664ba42e83fd8e89b8e72d90396" gracePeriod=30
Oct 31 21:41:04.061102 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:04.061070    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:04.245623 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:04.245557    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:04.327681 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:04.327650    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2"
Oct 31 21:41:04.357301 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:04.357259    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:04.549970 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:04.549594    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e8fc85a9-ec28-4322-a395-cc68f6a65947-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8fc85a9-ec28-4322-a395-cc68f6a65947-metrics-certs\") pod \"e8fc85a9-ec28-4322-a395-cc68f6a65947\" (UID: \"e8fc85a9-ec28-4322-a395-cc68f6a65947\") "
Oct 31 21:41:04.549970 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:04.549679    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e8fc85a9-ec28-4322-a395-cc68f6a65947-kube-api-access-rv4cm\" (UniqueName: \"kubernetes.io/projected/e8fc85a9-ec28-4322-a395-cc68f6a65947-kube-api-access-rv4cm\") pod \"e8fc85a9-ec28-4322-a395-cc68f6a65947\" (UID: \"e8fc85a9-ec28-4322-a395-cc68f6a65947\") "
Oct 31 21:41:04.549970 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:04.549768    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e8fc85a9-ec28-4322-a395-cc68f6a65947-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8fc85a9-ec28-4322-a395-cc68f6a65947-openshift-service-ca\") pod \"e8fc85a9-ec28-4322-a395-cc68f6a65947\" (UID: \"e8fc85a9-ec28-4322-a395-cc68f6a65947\") "
Oct 31 21:41:04.549970 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:04.549961    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e8fc85a9-ec28-4322-a395-cc68f6a65947-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8fc85a9-ec28-4322-a395-cc68f6a65947-maas-api-tls\") pod \"e8fc85a9-ec28-4322-a395-cc68f6a65947\" (UID: \"e8fc85a9-ec28-4322-a395-cc68f6a65947\") "
Oct 31 21:41:04.559205 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:04.559154    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8fc85a9-ec28-4322-a395-cc68f6a65947-openshift-service-ca" pod "e8fc85a9-ec28-4322-a395-cc68f6a65947" (UID: "e8fc85a9-ec28-4322-a395-cc68f6a65947"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:41:04.589573 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:04.580213    4460 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8fc85a9-ec28-4322-a395-cc68f6a65947-openshift-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:41:04.610996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:04.604684    4460 generic.go:356] "Generic (PLEG): container finished" podID="e8fc85a9-ec28-4322-a395-cc68f6a65947" containerID="f9fa14423d5896c7ce20f4d4857396fa8f079664ba42e83fd8e89b8e72d90396" exitCode=0
Oct 31 21:41:04.610996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:04.604992    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2" event={"ID":"e8fc85a9-ec28-4322-a395-cc68f6a65947","Type":"ContainerDied","Data":"f9fa14423d5896c7ce20f4d4857396fa8f079664ba42e83fd8e89b8e72d90396"}
Oct 31 21:41:04.610996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:04.605055    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2" event={"ID":"e8fc85a9-ec28-4322-a395-cc68f6a65947","Type":"ContainerDied","Data":"24539374cccbf63b8c55c52834fbbcd588ffe509b3a90d1435d518e6d08940e1"}
Oct 31 21:41:04.610996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:04.605089    4460 scope.go:122] "RemoveContainer" containerID="f9fa14423d5896c7ce20f4d4857396fa8f079664ba42e83fd8e89b8e72d90396"
Oct 31 21:41:04.610996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:04.605421    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2"
Oct 31 21:41:04.652984 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:04.647376    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8fc85a9-ec28-4322-a395-cc68f6a65947-metrics-certs" pod "e8fc85a9-ec28-4322-a395-cc68f6a65947" (UID: "e8fc85a9-ec28-4322-a395-cc68f6a65947"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 21:41:04.652984 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:04.649127    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8fc85a9-ec28-4322-a395-cc68f6a65947-kube-api-access-rv4cm" pod "e8fc85a9-ec28-4322-a395-cc68f6a65947" (UID: "e8fc85a9-ec28-4322-a395-cc68f6a65947"). InnerVolumeSpecName "kube-api-access-rv4cm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:41:04.722970 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:04.722424    4460 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8fc85a9-ec28-4322-a395-cc68f6a65947-metrics-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:41:04.722970 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:04.722484    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rv4cm\" (UniqueName: \"kubernetes.io/projected/e8fc85a9-ec28-4322-a395-cc68f6a65947-kube-api-access-rv4cm\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:41:04.751966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:04.750187    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8fc85a9-ec28-4322-a395-cc68f6a65947-maas-api-tls" pod "e8fc85a9-ec28-4322-a395-cc68f6a65947" (UID: "e8fc85a9-ec28-4322-a395-cc68f6a65947"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 21:41:04.766179 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:04.755062    4460 scope.go:122] "RemoveContainer" containerID="f9fa14423d5896c7ce20f4d4857396fa8f079664ba42e83fd8e89b8e72d90396"
Oct 31 21:41:04.766179 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:41:04.755511    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9fa14423d5896c7ce20f4d4857396fa8f079664ba42e83fd8e89b8e72d90396\": container with ID starting with f9fa14423d5896c7ce20f4d4857396fa8f079664ba42e83fd8e89b8e72d90396 not found: ID does not exist" containerID="f9fa14423d5896c7ce20f4d4857396fa8f079664ba42e83fd8e89b8e72d90396"
Oct 31 21:41:04.766179 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:04.755555    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9fa14423d5896c7ce20f4d4857396fa8f079664ba42e83fd8e89b8e72d90396"} err="failed to get container status \"f9fa14423d5896c7ce20f4d4857396fa8f079664ba42e83fd8e89b8e72d90396\": rpc error: code = NotFound desc = could not find container \"f9fa14423d5896c7ce20f4d4857396fa8f079664ba42e83fd8e89b8e72d90396\": container with ID starting with f9fa14423d5896c7ce20f4d4857396fa8f079664ba42e83fd8e89b8e72d90396 not found: ID does not exist"
Oct 31 21:41:04.824221 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:04.824133    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8fc85a9-ec28-4322-a395-cc68f6a65947-maas-api-tls\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:41:04.920614 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:04.920574    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:04.956505 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:04.956470    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2"]
Oct 31 21:41:04.971462 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:04.971428    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-89d4a2-76cddc79c7-42jb2"]
Oct 31 21:41:05.036580 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:05.036546    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:05.144525 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:05.144439    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:05.549788 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:05.549754    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:06.225703 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:06.225665    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e8fc85a9-ec28-4322-a395-cc68f6a65947" path="/var/lib/kubelet/pods/e8fc85a9-ec28-4322-a395-cc68f6a65947/volumes"
Oct 31 21:41:06.841264 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:06.841232    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:08.493119 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:08.493088    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-7b68fff8cd-t85fc"
Oct 31 21:41:08.493550 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:08.493184    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-6d947544ff-6kxwd"
Oct 31 21:41:08.555132 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:08.555082    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:08.849261 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:08.849204    4460 patch_prober.go:29] interesting pod/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 21:41:08.849460 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:08.849291    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td" podUID="0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 21:41:09.200740 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.200712    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:41:09.294031 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.293994    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istio-data\") pod \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") "
Oct 31 21:41:09.294210 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.294159    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istio-podinfo\") pod \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") "
Oct 31 21:41:09.294276 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.294214    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-workload-socket\") pod \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") "
Oct 31 21:41:09.294276 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.294248    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-workload-certs\") pod \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") "
Oct 31 21:41:09.294385 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.294288    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-credential-socket\") pod \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") "
Oct 31 21:41:09.294385 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.294322    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istio-token\" (UniqueName: \"kubernetes.io/projected/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istio-token\") pod \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") "
Oct 31 21:41:09.294385 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.294329    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istio-data" pod "0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9" (UID: "0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:41:09.294385 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.294362    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istiod-ca-cert\") pod \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") "
Oct 31 21:41:09.294662 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.294622    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-workload-certs" pod "0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9" (UID: "0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:41:09.294662 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.294636    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-credential-socket" pod "0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9" (UID: "0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:41:09.294849 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.294820    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-workload-socket" pod "0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9" (UID: "0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:41:09.295216 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.295126    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istiod-ca-cert" pod "0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9" (UID: "0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:41:09.295309 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.295238    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istio-envoy\") pod \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") "
Oct 31 21:41:09.295309 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.295299    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-kube-api-access-9vxwg\" (UniqueName: \"kubernetes.io/projected/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-kube-api-access-9vxwg\") pod \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\" (UID: \"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9\") "
Oct 31 21:41:09.296210 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.295935    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:41:09.296210 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.295978    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:41:09.296210 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.295994    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:41:09.296210 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.296009    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:41:09.296210 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.296022    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:41:09.297242 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.297214    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istio-podinfo" pod "0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9" (UID: "0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:41:09.297639 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.297614    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istio-token" pod "0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9" (UID: "0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:41:09.298570 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.298541    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-kube-api-access-9vxwg" pod "0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9" (UID: "0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9"). InnerVolumeSpecName "kube-api-access-9vxwg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:41:09.299338 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.299303    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istio-envoy" pod "0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9" (UID: "0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:41:09.397825 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.397722    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:41:09.397825 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.397758    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:41:09.397825 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.397770    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:41:09.397825 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.397784    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vxwg\" (UniqueName: \"kubernetes.io/projected/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9-kube-api-access-9vxwg\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:41:09.647158 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.647125    4460 generic.go:356] "Generic (PLEG): container finished" podID="0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9" containerID="4b3a8d0019acff71fa51bf90cc1b41c39d89266cf70bc5e29f7549c501fb5c73" exitCode=0
Oct 31 21:41:09.647618 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.647176    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td" event={"ID":"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9","Type":"ContainerDied","Data":"4b3a8d0019acff71fa51bf90cc1b41c39d89266cf70bc5e29f7549c501fb5c73"}
Oct 31 21:41:09.647618 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.647188    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"
Oct 31 21:41:09.647618 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.647228    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td" event={"ID":"0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9","Type":"ContainerDied","Data":"990f43e0eef8c639fe47912524b28b8e9e69e80398ded7fef28269b1ae468bac"}
Oct 31 21:41:09.647618 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.647260    4460 scope.go:122] "RemoveContainer" containerID="4b3a8d0019acff71fa51bf90cc1b41c39d89266cf70bc5e29f7549c501fb5c73"
Oct 31 21:41:09.660118 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.660092    4460 scope.go:122] "RemoveContainer" containerID="ec253cef5e4639cf518d7b9e22cfb51844f7860cee985b4d87e91546777fba30"
Oct 31 21:41:09.669886 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.669865    4460 scope.go:122] "RemoveContainer" containerID="4b3a8d0019acff71fa51bf90cc1b41c39d89266cf70bc5e29f7549c501fb5c73"
Oct 31 21:41:09.670233 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:41:09.670208    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b3a8d0019acff71fa51bf90cc1b41c39d89266cf70bc5e29f7549c501fb5c73\": container with ID starting with 4b3a8d0019acff71fa51bf90cc1b41c39d89266cf70bc5e29f7549c501fb5c73 not found: ID does not exist" containerID="4b3a8d0019acff71fa51bf90cc1b41c39d89266cf70bc5e29f7549c501fb5c73"
Oct 31 21:41:09.670294 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.670243    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b3a8d0019acff71fa51bf90cc1b41c39d89266cf70bc5e29f7549c501fb5c73"} err="failed to get container status \"4b3a8d0019acff71fa51bf90cc1b41c39d89266cf70bc5e29f7549c501fb5c73\": rpc error: code = NotFound desc = could not find container \"4b3a8d0019acff71fa51bf90cc1b41c39d89266cf70bc5e29f7549c501fb5c73\": container with ID starting with 4b3a8d0019acff71fa51bf90cc1b41c39d89266cf70bc5e29f7549c501fb5c73 not found: ID does not exist"
Oct 31 21:41:09.670294 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.670262    4460 scope.go:122] "RemoveContainer" containerID="ec253cef5e4639cf518d7b9e22cfb51844f7860cee985b4d87e91546777fba30"
Oct 31 21:41:09.670571 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:41:09.670547    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec253cef5e4639cf518d7b9e22cfb51844f7860cee985b4d87e91546777fba30\": container with ID starting with ec253cef5e4639cf518d7b9e22cfb51844f7860cee985b4d87e91546777fba30 not found: ID does not exist" containerID="ec253cef5e4639cf518d7b9e22cfb51844f7860cee985b4d87e91546777fba30"
Oct 31 21:41:09.670624 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.670582    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec253cef5e4639cf518d7b9e22cfb51844f7860cee985b4d87e91546777fba30"} err="failed to get container status \"ec253cef5e4639cf518d7b9e22cfb51844f7860cee985b4d87e91546777fba30\": rpc error: code = NotFound desc = could not find container \"ec253cef5e4639cf518d7b9e22cfb51844f7860cee985b4d87e91546777fba30\": container with ID starting with ec253cef5e4639cf518d7b9e22cfb51844f7860cee985b4d87e91546777fba30 not found: ID does not exist"
Oct 31 21:41:09.705363 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.705319    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"]
Oct 31 21:41:09.718696 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:09.718660    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-07afdd-openshift-default-6b44f5d885-r75td"]
Oct 31 21:41:10.228742 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:10.228705    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9" path="/var/lib/kubelet/pods/0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9/volumes"
Oct 31 21:41:10.719666 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:10.719623    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:12.063801 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:12.063754    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:12.175664 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:12.175622    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:12.398253 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:12.398153    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"]
Oct 31 21:41:12.398491 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:12.398456    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd" podUID="b49ec684-cafd-4063-9ffa-c3884fbe7abd" containerName="istio-proxy" containerID="cri-o://49a63fb4fe198f650749b8d65d097b9ff37d1864d463edc92bb93e5e9d8e3132" gracePeriod=30
Oct 31 21:41:12.424150 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:12.424109    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:12.757154 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:12.757056    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:15.048425 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:15.048379    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:15.321501 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:15.321398    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:15.469934 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:15.469899    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:16.751418 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:16.751376    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:17.690094 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.690068    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:41:17.737966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.737905    4460 generic.go:356] "Generic (PLEG): container finished" podID="b49ec684-cafd-4063-9ffa-c3884fbe7abd" containerID="49a63fb4fe198f650749b8d65d097b9ff37d1864d463edc92bb93e5e9d8e3132" exitCode=0
Oct 31 21:41:17.738155 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.737992    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"
Oct 31 21:41:17.738155 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.737996    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd" event={"ID":"b49ec684-cafd-4063-9ffa-c3884fbe7abd","Type":"ContainerDied","Data":"49a63fb4fe198f650749b8d65d097b9ff37d1864d463edc92bb93e5e9d8e3132"}
Oct 31 21:41:17.738155 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.738120    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd" event={"ID":"b49ec684-cafd-4063-9ffa-c3884fbe7abd","Type":"ContainerDied","Data":"6d690c8ece2811177d0eb0e1deb4a89b4dd0ff1c02476127912e1beb0cfb5b60"}
Oct 31 21:41:17.738155 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.738147    4460 scope.go:122] "RemoveContainer" containerID="49a63fb4fe198f650749b8d65d097b9ff37d1864d463edc92bb93e5e9d8e3132"
Oct 31 21:41:17.748826 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.748798    4460 scope.go:122] "RemoveContainer" containerID="c1073b04fdbe5c21ec9137e8bf45d09178688b7382975481c53333333a7d29b1"
Oct 31 21:41:17.764395 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.764337    4460 scope.go:122] "RemoveContainer" containerID="49a63fb4fe198f650749b8d65d097b9ff37d1864d463edc92bb93e5e9d8e3132"
Oct 31 21:41:17.764782 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:41:17.764756    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49a63fb4fe198f650749b8d65d097b9ff37d1864d463edc92bb93e5e9d8e3132\": container with ID starting with 49a63fb4fe198f650749b8d65d097b9ff37d1864d463edc92bb93e5e9d8e3132 not found: ID does not exist" containerID="49a63fb4fe198f650749b8d65d097b9ff37d1864d463edc92bb93e5e9d8e3132"
Oct 31 21:41:17.764845 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.764796    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49a63fb4fe198f650749b8d65d097b9ff37d1864d463edc92bb93e5e9d8e3132"} err="failed to get container status \"49a63fb4fe198f650749b8d65d097b9ff37d1864d463edc92bb93e5e9d8e3132\": rpc error: code = NotFound desc = could not find container \"49a63fb4fe198f650749b8d65d097b9ff37d1864d463edc92bb93e5e9d8e3132\": container with ID starting with 49a63fb4fe198f650749b8d65d097b9ff37d1864d463edc92bb93e5e9d8e3132 not found: ID does not exist"
Oct 31 21:41:17.764845 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.764820    4460 scope.go:122] "RemoveContainer" containerID="c1073b04fdbe5c21ec9137e8bf45d09178688b7382975481c53333333a7d29b1"
Oct 31 21:41:17.765199 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:41:17.765165    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1073b04fdbe5c21ec9137e8bf45d09178688b7382975481c53333333a7d29b1\": container with ID starting with c1073b04fdbe5c21ec9137e8bf45d09178688b7382975481c53333333a7d29b1 not found: ID does not exist" containerID="c1073b04fdbe5c21ec9137e8bf45d09178688b7382975481c53333333a7d29b1"
Oct 31 21:41:17.765270 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.765214    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1073b04fdbe5c21ec9137e8bf45d09178688b7382975481c53333333a7d29b1"} err="failed to get container status \"c1073b04fdbe5c21ec9137e8bf45d09178688b7382975481c53333333a7d29b1\": rpc error: code = NotFound desc = could not find container \"c1073b04fdbe5c21ec9137e8bf45d09178688b7382975481c53333333a7d29b1\": container with ID starting with c1073b04fdbe5c21ec9137e8bf45d09178688b7382975481c53333333a7d29b1 not found: ID does not exist"
Oct 31 21:41:17.795376 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.795343    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-workload-certs\") pod \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") "
Oct 31 21:41:17.795583 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.795389    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istio-data\") pod \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") "
Oct 31 21:41:17.795583 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.795479    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-workload-socket\") pod \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") "
Oct 31 21:41:17.795583 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.795577    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-credential-socket\") pod \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") "
Oct 31 21:41:17.795766 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.795689    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-workload-certs" pod "b49ec684-cafd-4063-9ffa-c3884fbe7abd" (UID: "b49ec684-cafd-4063-9ffa-c3884fbe7abd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:41:17.795825 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.795761    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istio-data" pod "b49ec684-cafd-4063-9ffa-c3884fbe7abd" (UID: "b49ec684-cafd-4063-9ffa-c3884fbe7abd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:41:17.795882 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.795852    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-workload-socket" pod "b49ec684-cafd-4063-9ffa-c3884fbe7abd" (UID: "b49ec684-cafd-4063-9ffa-c3884fbe7abd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:41:17.795882 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.795876    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-credential-socket" pod "b49ec684-cafd-4063-9ffa-c3884fbe7abd" (UID: "b49ec684-cafd-4063-9ffa-c3884fbe7abd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:41:17.796041 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.795908    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istio-podinfo\") pod \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") "
Oct 31 21:41:17.796041 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.795993    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istio-envoy\") pod \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") "
Oct 31 21:41:17.796041 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.796025    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istio-token\" (UniqueName: \"kubernetes.io/projected/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istio-token\") pod \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") "
Oct 31 21:41:17.796203 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.796045    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b49ec684-cafd-4063-9ffa-c3884fbe7abd-kube-api-access-d85g2\" (UniqueName: \"kubernetes.io/projected/b49ec684-cafd-4063-9ffa-c3884fbe7abd-kube-api-access-d85g2\") pod \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") "
Oct 31 21:41:17.796203 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.796137    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istiod-ca-cert\") pod \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\" (UID: \"b49ec684-cafd-4063-9ffa-c3884fbe7abd\") "
Oct 31 21:41:17.797014 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.796982    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istiod-ca-cert" pod "b49ec684-cafd-4063-9ffa-c3884fbe7abd" (UID: "b49ec684-cafd-4063-9ffa-c3884fbe7abd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:41:17.797748 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.797719    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:41:17.797879 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.797753    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:41:17.797879 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.797768    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:41:17.797879 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.797781    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:41:17.797879 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.797794    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:41:17.798730 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.798701    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istio-podinfo" pod "b49ec684-cafd-4063-9ffa-c3884fbe7abd" (UID: "b49ec684-cafd-4063-9ffa-c3884fbe7abd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:41:17.799012 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.798987    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istio-token" pod "b49ec684-cafd-4063-9ffa-c3884fbe7abd" (UID: "b49ec684-cafd-4063-9ffa-c3884fbe7abd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:41:17.799196 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.799174    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istio-envoy" pod "b49ec684-cafd-4063-9ffa-c3884fbe7abd" (UID: "b49ec684-cafd-4063-9ffa-c3884fbe7abd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:41:17.800084 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.800061    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b49ec684-cafd-4063-9ffa-c3884fbe7abd-kube-api-access-d85g2" pod "b49ec684-cafd-4063-9ffa-c3884fbe7abd" (UID: "b49ec684-cafd-4063-9ffa-c3884fbe7abd"). InnerVolumeSpecName "kube-api-access-d85g2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:41:17.899359 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.899316    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:41:17.899359 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.899352    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:41:17.899359 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.899360    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b49ec684-cafd-4063-9ffa-c3884fbe7abd-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:41:17.899606 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:17.899369    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d85g2\" (UniqueName: \"kubernetes.io/projected/b49ec684-cafd-4063-9ffa-c3884fbe7abd-kube-api-access-d85g2\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:41:18.089178 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:18.089143    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"]
Oct 31 21:41:18.093186 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:18.093158    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-89d4a2-openshift-default-6d779f946c-brfcd"]
Oct 31 21:41:18.227353 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:18.227310    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b49ec684-cafd-4063-9ffa-c3884fbe7abd" path="/var/lib/kubelet/pods/b49ec684-cafd-4063-9ffa-c3884fbe7abd/volumes"
Oct 31 21:41:18.474643 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:18.474552    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:19.027020 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:19.026986    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:41:20.225224 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:20.225190    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:21.454253 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:41:21.454214    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:41:22.171301 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:22.171265    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:24.422637 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:24.422600    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:26.150582 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:26.150541    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:28.074551 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:28.074502    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:28.258868 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:28.258822    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:28.368089 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:28.367982    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:28.696626 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:28.692632    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:28.815137 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:28.815091    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:28.973018 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:28.957647    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:29.359555 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:29.359520    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:30.723724 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:30.723688    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:32.569870 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:32.569837    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:34.833767 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:34.833730    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:35.928361 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:35.928324    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:36.037214 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:36.037175    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:36.248055 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:36.247923    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:36.356582 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:36.356538    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:36.359474 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:36.359445    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-752623-6bc5bf56b6-pm4z4"]
Oct 31 21:41:36.359690 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:36.359667    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-752623-6bc5bf56b6-pm4z4" podUID="c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356" containerName="payload-processing" containerID="cri-o://602b7206297a2f8f101f84a62d827f8e1890aaae6d6713893f61dd7660f47573" gracePeriod=30
Oct 31 21:41:36.467743 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:36.467701    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:36.580881 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:36.580838    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:36.673540 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:36.673512    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-752623-6bc5bf56b6-pm4z4"
Oct 31 21:41:36.748612 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:36.748575    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356-plugins-config-volume\") pod \"c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356\" (UID: \"c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356\") "
Oct 31 21:41:36.755269 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:36.755227    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356-kube-api-access-g6r4t\" (UniqueName: \"kubernetes.io/projected/c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356-kube-api-access-g6r4t\") pod \"c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356\" (UID: \"c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356\") "
Oct 31 21:41:36.764545 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:36.764476    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356-plugins-config-volume" pod "c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356" (UID: "c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:41:36.827977 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:36.817102    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356-kube-api-access-g6r4t" pod "c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356" (UID: "c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356"). InnerVolumeSpecName "kube-api-access-g6r4t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:41:36.861866 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:36.861820    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6r4t\" (UniqueName: \"kubernetes.io/projected/c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356-kube-api-access-g6r4t\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:41:36.862183 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:36.862159    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:41:36.945343 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:36.943548    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:36.953677 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:36.953639    4460 generic.go:356] "Generic (PLEG): container finished" podID="c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356" containerID="602b7206297a2f8f101f84a62d827f8e1890aaae6d6713893f61dd7660f47573" exitCode=0
Oct 31 21:41:36.954031 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:36.954003    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-752623-6bc5bf56b6-pm4z4" event={"ID":"c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356","Type":"ContainerDied","Data":"602b7206297a2f8f101f84a62d827f8e1890aaae6d6713893f61dd7660f47573"}
Oct 31 21:41:36.954224 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:36.954202    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-752623-6bc5bf56b6-pm4z4" event={"ID":"c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356","Type":"ContainerDied","Data":"5a925e3089a8709901072dc35915fda8257603c203bcf243e66891f2762e24ad"}
Oct 31 21:41:36.954364 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:36.954349    4460 scope.go:122] "RemoveContainer" containerID="602b7206297a2f8f101f84a62d827f8e1890aaae6d6713893f61dd7660f47573"
Oct 31 21:41:36.954646 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:36.954626    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-752623-6bc5bf56b6-pm4z4"
Oct 31 21:41:36.994840 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:36.994802    4460 scope.go:122] "RemoveContainer" containerID="602b7206297a2f8f101f84a62d827f8e1890aaae6d6713893f61dd7660f47573"
Oct 31 21:41:36.995848 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:41:36.995820    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"602b7206297a2f8f101f84a62d827f8e1890aaae6d6713893f61dd7660f47573\": container with ID starting with 602b7206297a2f8f101f84a62d827f8e1890aaae6d6713893f61dd7660f47573 not found: ID does not exist" containerID="602b7206297a2f8f101f84a62d827f8e1890aaae6d6713893f61dd7660f47573"
Oct 31 21:41:36.996037 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:36.996010    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"602b7206297a2f8f101f84a62d827f8e1890aaae6d6713893f61dd7660f47573"} err="failed to get container status \"602b7206297a2f8f101f84a62d827f8e1890aaae6d6713893f61dd7660f47573\": rpc error: code = NotFound desc = could not find container \"602b7206297a2f8f101f84a62d827f8e1890aaae6d6713893f61dd7660f47573\": container with ID starting with 602b7206297a2f8f101f84a62d827f8e1890aaae6d6713893f61dd7660f47573 not found: ID does not exist"
Oct 31 21:41:37.056583 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:37.056552    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:37.080714 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:37.080638    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-752623-6bc5bf56b6-pm4z4"]
Oct 31 21:41:37.083815 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:37.083792    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-752623-6bc5bf56b6-pm4z4"]
Oct 31 21:41:37.197960 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:37.193601    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:37.647010 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:37.646972    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:38.226509 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:38.226474    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356" path="/var/lib/kubelet/pods/c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356/volumes"
Oct 31 21:41:38.314231 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:38.314188    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf"]
Oct 31 21:41:38.314677 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:38.314648    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf" podUID="9091a526-1fb8-494b-a486-6c4e67dd9953" containerName="maas-api" containerID="cri-o://f2dbac6c121eafdb29baf0ee04d09221c8e8303dd02ce7548e5815c336c94d04" gracePeriod=30
Oct 31 21:41:38.598352 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:38.598324    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf"
Oct 31 21:41:38.697341 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:38.697308    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9091a526-1fb8-494b-a486-6c4e67dd9953-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9091a526-1fb8-494b-a486-6c4e67dd9953-maas-api-tls\") pod \"9091a526-1fb8-494b-a486-6c4e67dd9953\" (UID: \"9091a526-1fb8-494b-a486-6c4e67dd9953\") "
Oct 31 21:41:38.697513 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:38.697362    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9091a526-1fb8-494b-a486-6c4e67dd9953-kube-api-access-2r6lz\" (UniqueName: \"kubernetes.io/projected/9091a526-1fb8-494b-a486-6c4e67dd9953-kube-api-access-2r6lz\") pod \"9091a526-1fb8-494b-a486-6c4e67dd9953\" (UID: \"9091a526-1fb8-494b-a486-6c4e67dd9953\") "
Oct 31 21:41:38.697561 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:38.697524    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9091a526-1fb8-494b-a486-6c4e67dd9953-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9091a526-1fb8-494b-a486-6c4e67dd9953-metrics-certs\") pod \"9091a526-1fb8-494b-a486-6c4e67dd9953\" (UID: \"9091a526-1fb8-494b-a486-6c4e67dd9953\") "
Oct 31 21:41:38.697622 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:38.697608    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9091a526-1fb8-494b-a486-6c4e67dd9953-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9091a526-1fb8-494b-a486-6c4e67dd9953-openshift-service-ca\") pod \"9091a526-1fb8-494b-a486-6c4e67dd9953\" (UID: \"9091a526-1fb8-494b-a486-6c4e67dd9953\") "
Oct 31 21:41:38.698061 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:38.698019    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9091a526-1fb8-494b-a486-6c4e67dd9953-openshift-service-ca" pod "9091a526-1fb8-494b-a486-6c4e67dd9953" (UID: "9091a526-1fb8-494b-a486-6c4e67dd9953"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:41:38.698196 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:38.698178    4460 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9091a526-1fb8-494b-a486-6c4e67dd9953-openshift-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:41:38.699555 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:38.699525    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9091a526-1fb8-494b-a486-6c4e67dd9953-metrics-certs" pod "9091a526-1fb8-494b-a486-6c4e67dd9953" (UID: "9091a526-1fb8-494b-a486-6c4e67dd9953"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 21:41:38.699555 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:38.699532    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9091a526-1fb8-494b-a486-6c4e67dd9953-kube-api-access-2r6lz" pod "9091a526-1fb8-494b-a486-6c4e67dd9953" (UID: "9091a526-1fb8-494b-a486-6c4e67dd9953"). InnerVolumeSpecName "kube-api-access-2r6lz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:41:38.699696 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:38.699610    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9091a526-1fb8-494b-a486-6c4e67dd9953-maas-api-tls" pod "9091a526-1fb8-494b-a486-6c4e67dd9953" (UID: "9091a526-1fb8-494b-a486-6c4e67dd9953"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 21:41:38.800101 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:38.800067    4460 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9091a526-1fb8-494b-a486-6c4e67dd9953-metrics-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:41:38.800101 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:38.800099    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9091a526-1fb8-494b-a486-6c4e67dd9953-maas-api-tls\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:41:38.800101 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:38.800110    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2r6lz\" (UniqueName: \"kubernetes.io/projected/9091a526-1fb8-494b-a486-6c4e67dd9953-kube-api-access-2r6lz\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:41:38.969953 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:38.969904    4460 generic.go:356] "Generic (PLEG): container finished" podID="9091a526-1fb8-494b-a486-6c4e67dd9953" containerID="f2dbac6c121eafdb29baf0ee04d09221c8e8303dd02ce7548e5815c336c94d04" exitCode=0
Oct 31 21:41:38.970133 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:38.969987    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf"
Oct 31 21:41:38.970133 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:38.969988    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf" event={"ID":"9091a526-1fb8-494b-a486-6c4e67dd9953","Type":"ContainerDied","Data":"f2dbac6c121eafdb29baf0ee04d09221c8e8303dd02ce7548e5815c336c94d04"}
Oct 31 21:41:38.970133 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:38.970051    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf" event={"ID":"9091a526-1fb8-494b-a486-6c4e67dd9953","Type":"ContainerDied","Data":"a90275d5ca83f758571ea7277d80b5847e986650c74b1932fc24b8e13142afd4"}
Oct 31 21:41:38.970133 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:38.970084    4460 scope.go:122] "RemoveContainer" containerID="f2dbac6c121eafdb29baf0ee04d09221c8e8303dd02ce7548e5815c336c94d04"
Oct 31 21:41:38.981594 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:38.981576    4460 scope.go:122] "RemoveContainer" containerID="f2dbac6c121eafdb29baf0ee04d09221c8e8303dd02ce7548e5815c336c94d04"
Oct 31 21:41:38.981894 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:41:38.981852    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2dbac6c121eafdb29baf0ee04d09221c8e8303dd02ce7548e5815c336c94d04\": container with ID starting with f2dbac6c121eafdb29baf0ee04d09221c8e8303dd02ce7548e5815c336c94d04 not found: ID does not exist" containerID="f2dbac6c121eafdb29baf0ee04d09221c8e8303dd02ce7548e5815c336c94d04"
Oct 31 21:41:38.981985 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:38.981885    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2dbac6c121eafdb29baf0ee04d09221c8e8303dd02ce7548e5815c336c94d04"} err="failed to get container status \"f2dbac6c121eafdb29baf0ee04d09221c8e8303dd02ce7548e5815c336c94d04\": rpc error: code = NotFound desc = could not find container \"f2dbac6c121eafdb29baf0ee04d09221c8e8303dd02ce7548e5815c336c94d04\": container with ID starting with f2dbac6c121eafdb29baf0ee04d09221c8e8303dd02ce7548e5815c336c94d04 not found: ID does not exist"
Oct 31 21:41:39.014532 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:39.014488    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:39.029931 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:39.029896    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf"]
Oct 31 21:41:39.042366 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:39.042335    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-c744c0-75fbdcdd5-fglsf"]
Oct 31 21:41:40.227007 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:40.226973    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9091a526-1fb8-494b-a486-6c4e67dd9953" path="/var/lib/kubelet/pods/9091a526-1fb8-494b-a486-6c4e67dd9953/volumes"
Oct 31 21:41:40.845636 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:40.845605    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:42.465305 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:42.465267    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:44.258113 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:44.258073    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:44.413085 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:44.413050    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:44.527784 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:44.525558    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:44.758644 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:44.758610    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:46.285794 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.285756    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:46.423854 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.423813    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:46.624529 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.624449    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:46.714392 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.714348    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"]
Oct 31 21:41:46.715425 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.715395    4460 cpu_manager.go:414] "containerMap: removing container" podUID="0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9" containerName="istio-proxy"
Oct 31 21:41:46.715425 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.715423    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9" containerName="istio-proxy"
Oct 31 21:41:46.715634 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.715459    4460 cpu_manager.go:414] "containerMap: removing container" podUID="b49ec684-cafd-4063-9ffa-c3884fbe7abd" containerName="istio-proxy"
Oct 31 21:41:46.715634 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.715467    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b49ec684-cafd-4063-9ffa-c3884fbe7abd" containerName="istio-proxy"
Oct 31 21:41:46.715634 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.715490    4460 cpu_manager.go:414] "containerMap: removing container" podUID="9091a526-1fb8-494b-a486-6c4e67dd9953" containerName="maas-api"
Oct 31 21:41:46.715634 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.715498    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9091a526-1fb8-494b-a486-6c4e67dd9953" containerName="maas-api"
Oct 31 21:41:46.715634 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.715527    4460 cpu_manager.go:414] "containerMap: removing container" podUID="b49ec684-cafd-4063-9ffa-c3884fbe7abd" containerName="istio-proxy"
Oct 31 21:41:46.715634 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.715534    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b49ec684-cafd-4063-9ffa-c3884fbe7abd" containerName="istio-proxy"
Oct 31 21:41:46.715634 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.715557    4460 cpu_manager.go:414] "containerMap: removing container" podUID="e8fc85a9-ec28-4322-a395-cc68f6a65947" containerName="maas-api"
Oct 31 21:41:46.715634 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.715565    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e8fc85a9-ec28-4322-a395-cc68f6a65947" containerName="maas-api"
Oct 31 21:41:46.715634 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.715575    4460 cpu_manager.go:414] "containerMap: removing container" podUID="c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356" containerName="payload-processing"
Oct 31 21:41:46.715634 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.715582    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356" containerName="payload-processing"
Oct 31 21:41:46.715634 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.715611    4460 cpu_manager.go:414] "containerMap: removing container" podUID="0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9" containerName="istio-proxy"
Oct 31 21:41:46.715634 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.715619    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9" containerName="istio-proxy"
Oct 31 21:41:46.716244 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.715793    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="e8fc85a9-ec28-4322-a395-cc68f6a65947" containerName="maas-api"
Oct 31 21:41:46.716244 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.715809    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="c0c0a5d2-db98-4e2a-a7eb-61aabb8fa356" containerName="payload-processing"
Oct 31 21:41:46.716244 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.715826    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9" containerName="istio-proxy"
Oct 31 21:41:46.716244 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.715837    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d68aba6-92a7-4f62-bc8f-d1fd40f64ed9" containerName="istio-proxy"
Oct 31 21:41:46.716244 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.715848    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="b49ec684-cafd-4063-9ffa-c3884fbe7abd" containerName="istio-proxy"
Oct 31 21:41:46.716244 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.715860    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="9091a526-1fb8-494b-a486-6c4e67dd9953" containerName="maas-api"
Oct 31 21:41:46.716244 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.715872    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="b49ec684-cafd-4063-9ffa-c3884fbe7abd" containerName="istio-proxy"
Oct 31 21:41:46.721455 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.721429    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:41:46.744410 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.744374    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"]
Oct 31 21:41:46.752518 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.752477    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-a-6d6576-openshift-default-dockercfg-f467b\""
Oct 31 21:41:46.764969 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.764927    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:46.856500 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.856467    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qxs7\" (UniqueName: \"kubernetes.io/projected/1f5fd98d-9122-4102-b317-d51242d31d0e-kube-api-access-9qxs7\") pod \"e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") " pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:41:46.856687 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.856655    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-istio-data\") pod \"e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") " pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:41:46.856738 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.856695    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-credential-socket\") pod \"e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") " pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:41:46.856785 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.856761    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-istio-envoy\") pod \"e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") " pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:41:46.856842 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.856831    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f5fd98d-9122-4102-b317-d51242d31d0e-istio-token\") pod \"e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") " pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:41:46.856933 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.856921    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-workload-socket\") pod \"e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") " pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:41:46.856994 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.856962    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-workload-certs\") pod \"e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") " pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:41:46.857036 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.857008    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f5fd98d-9122-4102-b317-d51242d31d0e-istiod-ca-cert\") pod \"e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") " pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:41:46.857077 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.857060    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f5fd98d-9122-4102-b317-d51242d31d0e-istio-podinfo\") pod \"e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") " pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:41:46.879907 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.879823    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:46.962124 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.962078    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qxs7\" (UniqueName: \"kubernetes.io/projected/1f5fd98d-9122-4102-b317-d51242d31d0e-kube-api-access-9qxs7\") pod \"e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") " pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:41:46.962616 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.962567    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-istio-data\") pod \"e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") " pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:41:46.962846 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.962823    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-credential-socket\") pod \"e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") " pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:41:46.963059 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.963020    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-istio-envoy\") pod \"e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") " pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:41:46.963248 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.963223    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f5fd98d-9122-4102-b317-d51242d31d0e-istio-token\") pod \"e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") " pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:41:46.963496 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.963470    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-workload-socket\") pod \"e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") " pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:41:46.963650 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.963632    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-workload-certs\") pod \"e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") " pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:41:46.963778 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.963763    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f5fd98d-9122-4102-b317-d51242d31d0e-istiod-ca-cert\") pod \"e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") " pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:41:46.967961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.963954    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f5fd98d-9122-4102-b317-d51242d31d0e-istio-podinfo\") pod \"e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") " pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:41:46.977357 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.977329    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-istio-data\") pod \"e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") " pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:41:46.981039 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.979234    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-istio-envoy\") pod \"e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") " pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:41:46.984966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.979465    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-credential-socket\") pod \"e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") " pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:41:46.985158 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.980115    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-workload-socket\") pod \"e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") " pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:41:46.985255 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.980603    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-workload-certs\") pod \"e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") " pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:41:46.989012 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:46.981308    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f5fd98d-9122-4102-b317-d51242d31d0e-istiod-ca-cert\") pod \"e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") " pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:41:47.004967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:47.003254    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f5fd98d-9122-4102-b317-d51242d31d0e-istio-podinfo\") pod \"e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") " pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:41:47.032379 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:47.032344    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qxs7\" (UniqueName: \"kubernetes.io/projected/1f5fd98d-9122-4102-b317-d51242d31d0e-kube-api-access-9qxs7\") pod \"e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") " pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:41:47.050141 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:47.036610    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f5fd98d-9122-4102-b317-d51242d31d0e-istio-token\") pod \"e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") " pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:41:47.339657 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:47.339620    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:41:47.673226 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:47.673198    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"]
Oct 31 21:41:47.673949 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:41:47.673912    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f5fd98d_9122_4102_b317_d51242d31d0e.slice/crio-89135fe4b0de02fa7319b2869757eff3f6ee57248c37d22abf1444ecfb2cf72a WatchSource:0}: Error finding container 89135fe4b0de02fa7319b2869757eff3f6ee57248c37d22abf1444ecfb2cf72a: Status 404 returned error can't find the container with id 89135fe4b0de02fa7319b2869757eff3f6ee57248c37d22abf1444ecfb2cf72a
Oct 31 21:41:47.676515 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:47.676486    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:41:47.676588 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:47.676549    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:41:47.676588 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:47.676578    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:41:48.069020 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:48.068977    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj" event={"ID":"1f5fd98d-9122-4102-b317-d51242d31d0e","Type":"ContainerStarted","Data":"d0afd41ea2de72627ed621723c6cc15a3e378594eb25ebfe0cbe61d6cffaee12"}
Oct 31 21:41:48.069202 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:48.069051    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj" event={"ID":"1f5fd98d-9122-4102-b317-d51242d31d0e","Type":"ContainerStarted","Data":"89135fe4b0de02fa7319b2869757eff3f6ee57248c37d22abf1444ecfb2cf72a"}
Oct 31 21:41:48.118795 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:48.118742    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj" podStartSLOduration=2.118728811 podStartE2EDuration="2.118728811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:41:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:41:48.117310589 +0000 UTC m=+1548.033620019" watchObservedRunningTime="2026-10-31 21:41:48.118728811 +0000 UTC m=+1548.035038242"
Oct 31 21:41:48.248332 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:48.248294    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:48.336008 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:48.331723    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:41:48.344978 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:48.344907    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:41:48.377002 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:48.376497    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:41:49.077821 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:49.077724    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:41:49.078735 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:49.078715    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:41:50.317339 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:50.317306    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:50.540995 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:50.536436    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:50.679484 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:50.677763    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:50.983226 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:50.983133    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:51.095192 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:51.095156    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:51.216972 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:51.214870    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:51.454269 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:41:51.454229    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:41:51.571391 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:51.571357    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:52.289442 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:52.289411    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:52.409622 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:52.409584    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:52.916740 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:52.916696    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:54.761416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:54.761366    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:55.162091 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:55.161985    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2"]
Oct 31 21:41:55.196026 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:55.195982    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:56.287137 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:56.287108    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-hxh82_bb3b6872-a3ff-438d-b1ba-45fca4844b6e/payload-processing/0.log"
Oct 31 21:41:56.509353 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:56.509325    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-hxh82_bb3b6872-a3ff-438d-b1ba-45fca4844b6e/payload-processing/0.log"
Oct 31 21:41:57.115513 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:57.115468    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:57.747567 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:57.747514    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:57.857619 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:57.857579    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:58.107430 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:58.107373    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:58.936689 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:58.936626    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:59.057385 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.057346    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:59.423814 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.423781    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"]
Oct 31 21:41:59.428303 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.428277    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:41:59.445188 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.445160    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-d43b66-openshift-default-dockercfg-kpxsh\""
Oct 31 21:41:59.464634 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.464592    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"]
Oct 31 21:41:59.476007 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.475912    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:41:59.617974 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.577997    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b619644-e691-44de-889f-782d25668bca-istio-token\") pod \"e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") " pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:41:59.618460 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.618421    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b619644-e691-44de-889f-782d25668bca-istiod-ca-cert\") pod \"e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") " pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:41:59.619455 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.619429    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-istio-data\") pod \"e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") " pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:41:59.619690 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.619669    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppt9g\" (UniqueName: \"kubernetes.io/projected/2b619644-e691-44de-889f-782d25668bca-kube-api-access-ppt9g\") pod \"e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") " pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:41:59.619829 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.619814    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b619644-e691-44de-889f-782d25668bca-istio-podinfo\") pod \"e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") " pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:41:59.619969 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.619930    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-istio-envoy\") pod \"e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") " pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:41:59.620129 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.620113    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-workload-socket\") pod \"e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") " pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:41:59.620276 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.620261    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-credential-socket\") pod \"e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") " pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:41:59.620511 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.620494    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-workload-certs\") pod \"e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") " pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:41:59.722833 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.722733    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-workload-certs\") pod \"e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") " pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:41:59.723646 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.723621    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b619644-e691-44de-889f-782d25668bca-istio-token\") pod \"e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") " pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:41:59.738895 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.738845    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b619644-e691-44de-889f-782d25668bca-istiod-ca-cert\") pod \"e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") " pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:41:59.739334 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.739308    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-istio-data\") pod \"e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") " pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:41:59.739522 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.739507    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppt9g\" (UniqueName: \"kubernetes.io/projected/2b619644-e691-44de-889f-782d25668bca-kube-api-access-ppt9g\") pod \"e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") " pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:41:59.739651 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.739637    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b619644-e691-44de-889f-782d25668bca-istio-podinfo\") pod \"e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") " pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:41:59.739762 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.739745    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-istio-envoy\") pod \"e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") " pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:41:59.739934 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.739911    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-workload-socket\") pod \"e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") " pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:41:59.740514 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.740081    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-istio-data\") pod \"e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") " pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:41:59.740514 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.723466    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-workload-certs\") pod \"e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") " pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:41:59.740514 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.740139    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-credential-socket\") pod \"e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") " pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:41:59.740775 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.739808    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b619644-e691-44de-889f-782d25668bca-istiod-ca-cert\") pod \"e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") " pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:41:59.740886 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.740790    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-credential-socket\") pod \"e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") " pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:41:59.758639 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.758568    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-istio-envoy\") pod \"e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") " pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:41:59.769964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.764207    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-workload-socket\") pod \"e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") " pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:41:59.772981 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.772890    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b619644-e691-44de-889f-782d25668bca-istio-podinfo\") pod \"e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") " pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:41:59.796028 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.795953    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b619644-e691-44de-889f-782d25668bca-istio-token\") pod \"e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") " pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:41:59.796921 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.796867    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppt9g\" (UniqueName: \"kubernetes.io/projected/2b619644-e691-44de-889f-782d25668bca-kube-api-access-ppt9g\") pod \"e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") " pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:41:59.827288 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:41:59.827217    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:00.049079 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:00.049040    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:42:00.372808 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:00.372777    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"]
Oct 31 21:42:00.374231 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:42:00.374200    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b619644_e691_44de_889f_782d25668bca.slice/crio-f530cc25556359a712144bdc70f3fe1989d69ad833f2354c808bdff83f94ea56 WatchSource:0}: Error finding container f530cc25556359a712144bdc70f3fe1989d69ad833f2354c808bdff83f94ea56: Status 404 returned error can't find the container with id f530cc25556359a712144bdc70f3fe1989d69ad833f2354c808bdff83f94ea56
Oct 31 21:42:00.377044 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:00.377013    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:42:00.377147 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:00.377080    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:42:00.377147 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:00.377109    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:42:00.679168 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:00.678935    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:00.978073 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:00.977984    4460 scope.go:122] "RemoveContainer" containerID="bb74edf28465cc677ed9989d10111ea68f9f67fada1ecb18bf09667d3a5ecd47"
Oct 31 21:42:01.158294 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:01.158248    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:01.173433 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:01.172574    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648" event={"ID":"2b619644-e691-44de-889f-782d25668bca","Type":"ContainerStarted","Data":"ffe58be85d96ec3e16e38eb96c5ea214b280066966d7ebff1efe09da52ed886c"}
Oct 31 21:42:01.173433 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:01.172640    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648" event={"ID":"2b619644-e691-44de-889f-782d25668bca","Type":"ContainerStarted","Data":"f530cc25556359a712144bdc70f3fe1989d69ad833f2354c808bdff83f94ea56"}
Oct 31 21:42:01.284964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:01.273928    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648" podStartSLOduration=2.273906532 podStartE2EDuration="2.273906532s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:41:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:42:01.266653326 +0000 UTC m=+1561.182962758" watchObservedRunningTime="2026-10-31 21:42:01.273906532 +0000 UTC m=+1561.190215964"
Oct 31 21:42:02.049782 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.049749    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:42:02.056147 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.056112    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:42:02.179835 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.179795    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:42:02.180770 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.180742    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:42:02.329055 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.328950    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:02.440366 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.440329    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:02.748776 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.748682    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"]
Oct 31 21:42:02.755272 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.755238    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:02.778000 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.775139    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-a-1d8f7b-openshift-default-dockercfg-ztlxq\""
Oct 31 21:42:02.779385 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.779124    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"]
Oct 31 21:42:02.804641 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.804602    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-credential-socket\") pod \"e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") " pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:02.804859 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.804663    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3788ae03-d501-4b1f-89e2-c47c533ead04-istio-token\") pod \"e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") " pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:02.804859 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.804778    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-workload-certs\") pod \"e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") " pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:02.805011 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.804914    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrhg5\" (UniqueName: \"kubernetes.io/projected/3788ae03-d501-4b1f-89e2-c47c533ead04-kube-api-access-mrhg5\") pod \"e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") " pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:02.805097 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.805070    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3788ae03-d501-4b1f-89e2-c47c533ead04-istio-podinfo\") pod \"e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") " pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:02.805230 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.805215    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3788ae03-d501-4b1f-89e2-c47c533ead04-istiod-ca-cert\") pod \"e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") " pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:02.805346 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.805330    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-workload-socket\") pod \"e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") " pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:02.805482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.805458    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-istio-envoy\") pod \"e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") " pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:02.805553 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.805504    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-istio-data\") pod \"e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") " pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:02.817389 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.817357    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:02.920990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.913992    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3788ae03-d501-4b1f-89e2-c47c533ead04-istiod-ca-cert\") pod \"e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") " pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:02.920990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.914127    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-workload-socket\") pod \"e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") " pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:02.920990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.914293    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-istio-envoy\") pod \"e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") " pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:02.920990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.914357    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-istio-data\") pod \"e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") " pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:02.920990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.914429    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-credential-socket\") pod \"e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") " pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:02.920990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.914464    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3788ae03-d501-4b1f-89e2-c47c533ead04-istio-token\") pod \"e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") " pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:02.920990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.914493    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-workload-certs\") pod \"e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") " pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:02.920990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.914618    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrhg5\" (UniqueName: \"kubernetes.io/projected/3788ae03-d501-4b1f-89e2-c47c533ead04-kube-api-access-mrhg5\") pod \"e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") " pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:02.920990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.914674    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3788ae03-d501-4b1f-89e2-c47c533ead04-istio-podinfo\") pod \"e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") " pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:02.932603 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.932556    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-istio-data\") pod \"e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") " pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:02.945107 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.945054    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3788ae03-d501-4b1f-89e2-c47c533ead04-istiod-ca-cert\") pod \"e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") " pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:02.962966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.949449    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-credential-socket\") pod \"e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") " pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:02.962966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.950089    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-workload-certs\") pod \"e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") " pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:02.962966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.950819    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3788ae03-d501-4b1f-89e2-c47c533ead04-istio-podinfo\") pod \"e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") " pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:02.962966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.951080    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-workload-socket\") pod \"e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") " pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:02.977523 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.977383    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-istio-envoy\") pod \"e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") " pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:02.999023 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.996788    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3788ae03-d501-4b1f-89e2-c47c533ead04-istio-token\") pod \"e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") " pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:02.999023 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:02.997958    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrhg5\" (UniqueName: \"kubernetes.io/projected/3788ae03-d501-4b1f-89e2-c47c533ead04-kube-api-access-mrhg5\") pod \"e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") " pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:03.089269 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:03.089227    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:03.125055 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:03.122809    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:03.479068 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:03.479024    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"]
Oct 31 21:42:03.480056 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:42:03.480025    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3788ae03_d501_4b1f_89e2_c47c533ead04.slice/crio-fa2d46065abd4ff70c664284e06494bcb6bbe2f37bde485a780209f429a75910 WatchSource:0}: Error finding container fa2d46065abd4ff70c664284e06494bcb6bbe2f37bde485a780209f429a75910: Status 404 returned error can't find the container with id fa2d46065abd4ff70c664284e06494bcb6bbe2f37bde485a780209f429a75910
Oct 31 21:42:03.482917 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:03.482882    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:42:03.483045 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:03.482988    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:42:03.483122 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:03.483040    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:42:03.597632 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:03.597582    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:03.739022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:03.738915    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:04.197236 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:04.197198    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9" event={"ID":"3788ae03-d501-4b1f-89e2-c47c533ead04","Type":"ContainerStarted","Data":"7dbb1b345324e0163f7ab4bc458a9797dae160c24aa274230203fa861d6b0197"}
Oct 31 21:42:04.197437 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:04.197257    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9" event={"ID":"3788ae03-d501-4b1f-89e2-c47c533ead04","Type":"ContainerStarted","Data":"fa2d46065abd4ff70c664284e06494bcb6bbe2f37bde485a780209f429a75910"}
Oct 31 21:42:04.261209 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:04.261155    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9" podStartSLOduration=2.261138214 podStartE2EDuration="2.261138214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:42:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:42:04.259695376 +0000 UTC m=+1564.176004805" watchObservedRunningTime="2026-10-31 21:42:04.261138214 +0000 UTC m=+1564.177447682"
Oct 31 21:42:04.470000 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:04.469828    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:05.090047 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:05.090009    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:05.096133 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:05.096102    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:05.204680 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:05.204539    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:05.206327 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:05.206298    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:05.315011 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:05.313389    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:05.435316 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:05.435216    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:06.002452 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:06.002263    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:06.099070 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:06.099032    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-7b68fff8cd-t85fc"]
Oct 31 21:42:06.099475 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:06.099302    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-7b68fff8cd-t85fc" podUID="035ef563-8474-46f9-9f37-0284c7f4721e" containerName="payload-processing" containerID="cri-o://6717754983b8c4daf7226d2f822cf2890c831e12a2966d6d1fc6c5bcf47c6945" gracePeriod=30
Oct 31 21:42:06.126484 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:06.126440    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-6d947544ff-6kxwd"]
Oct 31 21:42:06.126994 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:06.126964    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-6d947544ff-6kxwd" podUID="180d2d9e-b101-45fc-a814-368a8efab6f7" containerName="payload-pre-processing" containerID="cri-o://bb3526e6aa28c48baf28496123b22e53cf9db5d72651f7855eccd7dd2ae99b07" gracePeriod=30
Oct 31 21:42:06.183908 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:06.183754    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:06.300321 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:06.300286    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:06.475754 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:06.475644    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-7b68fff8cd-t85fc"
Oct 31 21:42:06.491965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:06.478847    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-6d947544ff-6kxwd"
Oct 31 21:42:06.607671 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:06.606327    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:06.637964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:06.627624    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/180d2d9e-b101-45fc-a814-368a8efab6f7-kube-api-access-4kzqb\" (UniqueName: \"kubernetes.io/projected/180d2d9e-b101-45fc-a814-368a8efab6f7-kube-api-access-4kzqb\") pod \"180d2d9e-b101-45fc-a814-368a8efab6f7\" (UID: \"180d2d9e-b101-45fc-a814-368a8efab6f7\") "
Oct 31 21:42:06.638310 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:06.638272    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/035ef563-8474-46f9-9f37-0284c7f4721e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/035ef563-8474-46f9-9f37-0284c7f4721e-plugins-config-volume\") pod \"035ef563-8474-46f9-9f37-0284c7f4721e\" (UID: \"035ef563-8474-46f9-9f37-0284c7f4721e\") "
Oct 31 21:42:06.653146 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:06.653105    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/035ef563-8474-46f9-9f37-0284c7f4721e-kube-api-access-qpkgg\" (UniqueName: \"kubernetes.io/projected/035ef563-8474-46f9-9f37-0284c7f4721e-kube-api-access-qpkgg\") pod \"035ef563-8474-46f9-9f37-0284c7f4721e\" (UID: \"035ef563-8474-46f9-9f37-0284c7f4721e\") "
Oct 31 21:42:06.653479 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:06.653461    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/180d2d9e-b101-45fc-a814-368a8efab6f7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/180d2d9e-b101-45fc-a814-368a8efab6f7-plugins-config-volume\") pod \"180d2d9e-b101-45fc-a814-368a8efab6f7\" (UID: \"180d2d9e-b101-45fc-a814-368a8efab6f7\") "
Oct 31 21:42:06.675963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:06.641437    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/035ef563-8474-46f9-9f37-0284c7f4721e-plugins-config-volume" pod "035ef563-8474-46f9-9f37-0284c7f4721e" (UID: "035ef563-8474-46f9-9f37-0284c7f4721e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:42:06.700961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:06.695273    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/180d2d9e-b101-45fc-a814-368a8efab6f7-plugins-config-volume" pod "180d2d9e-b101-45fc-a814-368a8efab6f7" (UID: "180d2d9e-b101-45fc-a814-368a8efab6f7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:42:06.721028 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:06.719143    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/180d2d9e-b101-45fc-a814-368a8efab6f7-kube-api-access-4kzqb" pod "180d2d9e-b101-45fc-a814-368a8efab6f7" (UID: "180d2d9e-b101-45fc-a814-368a8efab6f7"). InnerVolumeSpecName "kube-api-access-4kzqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:42:06.723396 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:06.723328    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/035ef563-8474-46f9-9f37-0284c7f4721e-kube-api-access-qpkgg" pod "035ef563-8474-46f9-9f37-0284c7f4721e" (UID: "035ef563-8474-46f9-9f37-0284c7f4721e"). InnerVolumeSpecName "kube-api-access-qpkgg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:42:06.763978 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:06.757220    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4kzqb\" (UniqueName: \"kubernetes.io/projected/180d2d9e-b101-45fc-a814-368a8efab6f7-kube-api-access-4kzqb\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:42:06.764233 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:06.764206    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/035ef563-8474-46f9-9f37-0284c7f4721e-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:42:06.764396 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:06.764379    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpkgg\" (UniqueName: \"kubernetes.io/projected/035ef563-8474-46f9-9f37-0284c7f4721e-kube-api-access-qpkgg\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:42:06.764514 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:06.764499    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/180d2d9e-b101-45fc-a814-368a8efab6f7-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:42:06.764604 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:06.763407    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:06.919080 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:06.912807    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:07.028277 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:07.028246    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:07.220108 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:07.220015    4460 generic.go:356] "Generic (PLEG): container finished" podID="180d2d9e-b101-45fc-a814-368a8efab6f7" containerID="bb3526e6aa28c48baf28496123b22e53cf9db5d72651f7855eccd7dd2ae99b07" exitCode=0
Oct 31 21:42:07.220108 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:07.220059    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-6d947544ff-6kxwd"
Oct 31 21:42:07.220609 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:07.220112    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-6d947544ff-6kxwd" event={"ID":"180d2d9e-b101-45fc-a814-368a8efab6f7","Type":"ContainerDied","Data":"bb3526e6aa28c48baf28496123b22e53cf9db5d72651f7855eccd7dd2ae99b07"}
Oct 31 21:42:07.220609 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:07.220164    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-6d947544ff-6kxwd" event={"ID":"180d2d9e-b101-45fc-a814-368a8efab6f7","Type":"ContainerDied","Data":"29fc1561bdb1d2f6efb31815eb0e69156b0011c02b72d336bc8da99d0e446c99"}
Oct 31 21:42:07.220609 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:07.220186    4460 scope.go:122] "RemoveContainer" containerID="bb3526e6aa28c48baf28496123b22e53cf9db5d72651f7855eccd7dd2ae99b07"
Oct 31 21:42:07.221889 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:07.221866    4460 generic.go:356] "Generic (PLEG): container finished" podID="035ef563-8474-46f9-9f37-0284c7f4721e" containerID="6717754983b8c4daf7226d2f822cf2890c831e12a2966d6d1fc6c5bcf47c6945" exitCode=0
Oct 31 21:42:07.222006 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:07.221910    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-7b68fff8cd-t85fc"
Oct 31 21:42:07.222064 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:07.222005    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-7b68fff8cd-t85fc" event={"ID":"035ef563-8474-46f9-9f37-0284c7f4721e","Type":"ContainerDied","Data":"6717754983b8c4daf7226d2f822cf2890c831e12a2966d6d1fc6c5bcf47c6945"}
Oct 31 21:42:07.222106 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:07.222059    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-7b68fff8cd-t85fc" event={"ID":"035ef563-8474-46f9-9f37-0284c7f4721e","Type":"ContainerDied","Data":"5eb365629feb5d57b68e9a53bc2807047dc9ce906fb5656b1992d6d90addba46"}
Oct 31 21:42:07.231544 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:07.231397    4460 scope.go:122] "RemoveContainer" containerID="bb3526e6aa28c48baf28496123b22e53cf9db5d72651f7855eccd7dd2ae99b07"
Oct 31 21:42:07.231728 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:42:07.231680    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb3526e6aa28c48baf28496123b22e53cf9db5d72651f7855eccd7dd2ae99b07\": container with ID starting with bb3526e6aa28c48baf28496123b22e53cf9db5d72651f7855eccd7dd2ae99b07 not found: ID does not exist" containerID="bb3526e6aa28c48baf28496123b22e53cf9db5d72651f7855eccd7dd2ae99b07"
Oct 31 21:42:07.231859 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:07.231720    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb3526e6aa28c48baf28496123b22e53cf9db5d72651f7855eccd7dd2ae99b07"} err="failed to get container status \"bb3526e6aa28c48baf28496123b22e53cf9db5d72651f7855eccd7dd2ae99b07\": rpc error: code = NotFound desc = could not find container \"bb3526e6aa28c48baf28496123b22e53cf9db5d72651f7855eccd7dd2ae99b07\": container with ID starting with bb3526e6aa28c48baf28496123b22e53cf9db5d72651f7855eccd7dd2ae99b07 not found: ID does not exist"
Oct 31 21:42:07.231859 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:07.231745    4460 scope.go:122] "RemoveContainer" containerID="6717754983b8c4daf7226d2f822cf2890c831e12a2966d6d1fc6c5bcf47c6945"
Oct 31 21:42:07.241133 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:07.241116    4460 scope.go:122] "RemoveContainer" containerID="6717754983b8c4daf7226d2f822cf2890c831e12a2966d6d1fc6c5bcf47c6945"
Oct 31 21:42:07.241385 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:42:07.241369    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6717754983b8c4daf7226d2f822cf2890c831e12a2966d6d1fc6c5bcf47c6945\": container with ID starting with 6717754983b8c4daf7226d2f822cf2890c831e12a2966d6d1fc6c5bcf47c6945 not found: ID does not exist" containerID="6717754983b8c4daf7226d2f822cf2890c831e12a2966d6d1fc6c5bcf47c6945"
Oct 31 21:42:07.241425 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:07.241392    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6717754983b8c4daf7226d2f822cf2890c831e12a2966d6d1fc6c5bcf47c6945"} err="failed to get container status \"6717754983b8c4daf7226d2f822cf2890c831e12a2966d6d1fc6c5bcf47c6945\": rpc error: code = NotFound desc = could not find container \"6717754983b8c4daf7226d2f822cf2890c831e12a2966d6d1fc6c5bcf47c6945\": container with ID starting with 6717754983b8c4daf7226d2f822cf2890c831e12a2966d6d1fc6c5bcf47c6945 not found: ID does not exist"
Oct 31 21:42:07.269423 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:07.269386    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:07.290468 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:07.290438    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-7b68fff8cd-t85fc"]
Oct 31 21:42:07.294792 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:07.294766    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-9bbfc5-7b68fff8cd-t85fc"]
Oct 31 21:42:07.331816 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:07.331785    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-6d947544ff-6kxwd"]
Oct 31 21:42:07.336829 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:07.336800    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-9bbfc5-6d947544ff-6kxwd"]
Oct 31 21:42:08.227527 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:08.227490    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="035ef563-8474-46f9-9f37-0284c7f4721e" path="/var/lib/kubelet/pods/035ef563-8474-46f9-9f37-0284c7f4721e/volumes"
Oct 31 21:42:08.228042 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:08.228024    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="180d2d9e-b101-45fc-a814-368a8efab6f7" path="/var/lib/kubelet/pods/180d2d9e-b101-45fc-a814-368a8efab6f7/volumes"
Oct 31 21:42:08.637676 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:08.637549    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:08.752716 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:08.752680    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:08.866378 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:08.866343    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:10.185086 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:10.185045    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2" podUID="d183766a-a405-445c-975a-06efb0a90fed" containerName="main" containerID="cri-o://9339962abf5d19decca54eb788c8dcc078e4e3d634b1f92652f73b0a7c951523" gracePeriod=45
Oct 31 21:42:10.486918 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:10.486885    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2"
Oct 31 21:42:10.488714 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:10.488679    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:10.596205 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:10.592833    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d183766a-a405-445c-975a-06efb0a90fed-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d183766a-a405-445c-975a-06efb0a90fed-tmp-dir\") pod \"d183766a-a405-445c-975a-06efb0a90fed\" (UID: \"d183766a-a405-445c-975a-06efb0a90fed\") "
Oct 31 21:42:10.596205 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:10.592905    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d183766a-a405-445c-975a-06efb0a90fed-kube-api-access-9b7tb\" (UniqueName: \"kubernetes.io/projected/d183766a-a405-445c-975a-06efb0a90fed-kube-api-access-9b7tb\") pod \"d183766a-a405-445c-975a-06efb0a90fed\" (UID: \"d183766a-a405-445c-975a-06efb0a90fed\") "
Oct 31 21:42:10.596205 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:10.593042    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d183766a-a405-445c-975a-06efb0a90fed-tls-certs\" (UniqueName: \"kubernetes.io/secret/d183766a-a405-445c-975a-06efb0a90fed-tls-certs\") pod \"d183766a-a405-445c-975a-06efb0a90fed\" (UID: \"d183766a-a405-445c-975a-06efb0a90fed\") "
Oct 31 21:42:10.596205 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:10.593087    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d183766a-a405-445c-975a-06efb0a90fed-dshm\" (UniqueName: \"kubernetes.io/empty-dir/d183766a-a405-445c-975a-06efb0a90fed-dshm\") pod \"d183766a-a405-445c-975a-06efb0a90fed\" (UID: \"d183766a-a405-445c-975a-06efb0a90fed\") "
Oct 31 21:42:10.596205 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:10.593193    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d183766a-a405-445c-975a-06efb0a90fed-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d183766a-a405-445c-975a-06efb0a90fed-model-cache\") pod \"d183766a-a405-445c-975a-06efb0a90fed\" (UID: \"d183766a-a405-445c-975a-06efb0a90fed\") "
Oct 31 21:42:10.596205 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:10.593241    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d183766a-a405-445c-975a-06efb0a90fed-home\" (UniqueName: \"kubernetes.io/empty-dir/d183766a-a405-445c-975a-06efb0a90fed-home\") pod \"d183766a-a405-445c-975a-06efb0a90fed\" (UID: \"d183766a-a405-445c-975a-06efb0a90fed\") "
Oct 31 21:42:10.640962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:10.615278    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d183766a-a405-445c-975a-06efb0a90fed-kube-api-access-9b7tb" pod "d183766a-a405-445c-975a-06efb0a90fed" (UID: "d183766a-a405-445c-975a-06efb0a90fed"). InnerVolumeSpecName "kube-api-access-9b7tb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:42:10.640962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:10.615505    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d183766a-a405-445c-975a-06efb0a90fed-tmp-dir" pod "d183766a-a405-445c-975a-06efb0a90fed" (UID: "d183766a-a405-445c-975a-06efb0a90fed"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:42:10.640962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:10.618053    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d183766a-a405-445c-975a-06efb0a90fed-home" pod "d183766a-a405-445c-975a-06efb0a90fed" (UID: "d183766a-a405-445c-975a-06efb0a90fed"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:42:10.640962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:10.618089    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d183766a-a405-445c-975a-06efb0a90fed-model-cache" pod "d183766a-a405-445c-975a-06efb0a90fed" (UID: "d183766a-a405-445c-975a-06efb0a90fed"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:42:10.640962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:10.629664    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d183766a-a405-445c-975a-06efb0a90fed-dshm" pod "d183766a-a405-445c-975a-06efb0a90fed" (UID: "d183766a-a405-445c-975a-06efb0a90fed"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:42:10.640962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:10.633522    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d183766a-a405-445c-975a-06efb0a90fed-tls-certs" pod "d183766a-a405-445c-975a-06efb0a90fed" (UID: "d183766a-a405-445c-975a-06efb0a90fed"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 21:42:10.695019 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:10.694977    4460 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d183766a-a405-445c-975a-06efb0a90fed-tls-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:42:10.695019 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:10.695011    4460 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d183766a-a405-445c-975a-06efb0a90fed-dshm\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:42:10.695019 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:10.695023    4460 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d183766a-a405-445c-975a-06efb0a90fed-model-cache\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:42:10.695271 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:10.695034    4460 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d183766a-a405-445c-975a-06efb0a90fed-home\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:42:10.695271 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:10.695070    4460 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d183766a-a405-445c-975a-06efb0a90fed-tmp-dir\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:42:10.695271 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:10.695082    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9b7tb\" (UniqueName: \"kubernetes.io/projected/d183766a-a405-445c-975a-06efb0a90fed-kube-api-access-9b7tb\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:42:11.251849 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:11.251812    4460 generic.go:356] "Generic (PLEG): container finished" podID="d183766a-a405-445c-975a-06efb0a90fed" containerID="9339962abf5d19decca54eb788c8dcc078e4e3d634b1f92652f73b0a7c951523" exitCode=0
Oct 31 21:42:11.252372 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:11.251863    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2" event={"ID":"d183766a-a405-445c-975a-06efb0a90fed","Type":"ContainerDied","Data":"9339962abf5d19decca54eb788c8dcc078e4e3d634b1f92652f73b0a7c951523"}
Oct 31 21:42:11.252372 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:11.251890    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2" event={"ID":"d183766a-a405-445c-975a-06efb0a90fed","Type":"ContainerDied","Data":"d9f13d48e8500fe665b640530b08d7206cedef0c9b647d99f99c7cbb9393b71e"}
Oct 31 21:42:11.252372 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:11.251906    4460 scope.go:122] "RemoveContainer" containerID="9339962abf5d19decca54eb788c8dcc078e4e3d634b1f92652f73b0a7c951523"
Oct 31 21:42:11.252372 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:11.251863    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2"
Oct 31 21:42:11.262497 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:11.262480    4460 scope.go:122] "RemoveContainer" containerID="9339962abf5d19decca54eb788c8dcc078e4e3d634b1f92652f73b0a7c951523"
Oct 31 21:42:11.262781 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:42:11.262763    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9339962abf5d19decca54eb788c8dcc078e4e3d634b1f92652f73b0a7c951523\": container with ID starting with 9339962abf5d19decca54eb788c8dcc078e4e3d634b1f92652f73b0a7c951523 not found: ID does not exist" containerID="9339962abf5d19decca54eb788c8dcc078e4e3d634b1f92652f73b0a7c951523"
Oct 31 21:42:11.262832 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:11.262791    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9339962abf5d19decca54eb788c8dcc078e4e3d634b1f92652f73b0a7c951523"} err="failed to get container status \"9339962abf5d19decca54eb788c8dcc078e4e3d634b1f92652f73b0a7c951523\": rpc error: code = NotFound desc = could not find container \"9339962abf5d19decca54eb788c8dcc078e4e3d634b1f92652f73b0a7c951523\": container with ID starting with 9339962abf5d19decca54eb788c8dcc078e4e3d634b1f92652f73b0a7c951523 not found: ID does not exist"
Oct 31 21:42:11.309449 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:11.309416    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2"]
Oct 31 21:42:11.322479 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:11.322449    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w4-f1e791/tenantcfg-model-f1e791-kserve-758b579b8f-86xs2"]
Oct 31 21:42:12.234951 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:12.234902    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d183766a-a405-445c-975a-06efb0a90fed" path="/var/lib/kubelet/pods/d183766a-a405-445c-975a-06efb0a90fed/volumes"
Oct 31 21:42:12.323565 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:12.323530    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:14.041924 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:14.041838    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:14.154510 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:14.154471    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:14.414179 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:14.414098    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"]
Oct 31 21:42:14.414396 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:14.414366    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k" podUID="d420b3f8-f5f8-4bd3-86b9-c4b6b6911041" containerName="istio-proxy" containerID="cri-o://b5396875d149dbf3ccc4a5d3e536abc90adea3c634dd53f17908910da3a26364" gracePeriod=30
Oct 31 21:42:14.455425 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:14.455380    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:15.990791 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:15.990760    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:16.206320 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.206283    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:16.422098 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.422059    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"]
Oct 31 21:42:16.422850 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.422826    4460 cpu_manager.go:414] "containerMap: removing container" podUID="035ef563-8474-46f9-9f37-0284c7f4721e" containerName="payload-processing"
Oct 31 21:42:16.422850 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.422847    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="035ef563-8474-46f9-9f37-0284c7f4721e" containerName="payload-processing"
Oct 31 21:42:16.423080 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.422878    4460 cpu_manager.go:414] "containerMap: removing container" podUID="d183766a-a405-445c-975a-06efb0a90fed" containerName="main"
Oct 31 21:42:16.423080 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.422884    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d183766a-a405-445c-975a-06efb0a90fed" containerName="main"
Oct 31 21:42:16.423080 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.422920    4460 cpu_manager.go:414] "containerMap: removing container" podUID="180d2d9e-b101-45fc-a814-368a8efab6f7" containerName="payload-pre-processing"
Oct 31 21:42:16.423080 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.422926    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="180d2d9e-b101-45fc-a814-368a8efab6f7" containerName="payload-pre-processing"
Oct 31 21:42:16.423305 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.423096    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="180d2d9e-b101-45fc-a814-368a8efab6f7" containerName="payload-pre-processing"
Oct 31 21:42:16.423305 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.423113    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="035ef563-8474-46f9-9f37-0284c7f4721e" containerName="payload-processing"
Oct 31 21:42:16.423305 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.423125    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="d183766a-a405-445c-975a-06efb0a90fed" containerName="main"
Oct 31 21:42:16.428168 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.428148    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:42:16.453505 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.453479    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"]
Oct 31 21:42:16.454294 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.454269    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-a-80542b-openshift-default-dockercfg-s5txp\""
Oct 31 21:42:16.487966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.487749    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:16.489013 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.488976    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-istio-data\") pod \"e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") " pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:42:16.489176 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.489105    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/928636a0-a99a-43da-b29f-6ad518f022e1-istio-token\") pod \"e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") " pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:42:16.489176 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.489148    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-854dp\" (UniqueName: \"kubernetes.io/projected/928636a0-a99a-43da-b29f-6ad518f022e1-kube-api-access-854dp\") pod \"e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") " pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:42:16.489302 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.489192    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-credential-socket\") pod \"e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") " pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:42:16.489302 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.489236    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-workload-socket\") pod \"e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") " pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:42:16.489302 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.489276    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-workload-certs\") pod \"e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") " pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:42:16.491648 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.489475    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/928636a0-a99a-43da-b29f-6ad518f022e1-istio-podinfo\") pod \"e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") " pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:42:16.491648 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.489660    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-istio-envoy\") pod \"e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") " pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:42:16.491648 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.489739    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/928636a0-a99a-43da-b29f-6ad518f022e1-istiod-ca-cert\") pod \"e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") " pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:42:16.498984 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.498929    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:42:16.633389 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.633353    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/928636a0-a99a-43da-b29f-6ad518f022e1-istio-token\") pod \"e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") " pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:42:16.667203 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.667158    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-854dp\" (UniqueName: \"kubernetes.io/projected/928636a0-a99a-43da-b29f-6ad518f022e1-kube-api-access-854dp\") pod \"e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") " pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:42:16.667730 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.667704    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-credential-socket\") pod \"e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") " pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:42:16.669058 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.669031    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-workload-socket\") pod \"e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") " pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:42:16.676756 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.676636    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-workload-certs\") pod \"e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") " pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:42:16.681575 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.681531    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/928636a0-a99a-43da-b29f-6ad518f022e1-istio-podinfo\") pod \"e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") " pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:42:16.681981 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.681961    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-istio-envoy\") pod \"e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") " pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:42:16.682138 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.682124    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/928636a0-a99a-43da-b29f-6ad518f022e1-istiod-ca-cert\") pod \"e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") " pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:42:16.683494 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.683470    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-istio-data\") pod \"e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") " pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:42:16.685260 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.685238    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-istio-data\") pod \"e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") " pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:42:16.686274 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.645755    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:16.687219 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.687095    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-credential-socket\") pod \"e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") " pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:42:16.687708 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.687656    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-workload-socket\") pod \"e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") " pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:42:16.688189 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.688151    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-workload-certs\") pod \"e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") " pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:42:16.689125 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.689085    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/928636a0-a99a-43da-b29f-6ad518f022e1-istio-token\") pod \"e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") " pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:42:16.691455 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.691430    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/928636a0-a99a-43da-b29f-6ad518f022e1-istiod-ca-cert\") pod \"e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") " pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:42:16.715962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.713048    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-854dp\" (UniqueName: \"kubernetes.io/projected/928636a0-a99a-43da-b29f-6ad518f022e1-kube-api-access-854dp\") pod \"e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") " pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:42:16.762698 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.758748    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/928636a0-a99a-43da-b29f-6ad518f022e1-istio-podinfo\") pod \"e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") " pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:42:16.762698 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:16.759354    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-istio-envoy\") pod \"e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") " pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:42:17.052466 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:17.052430    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:42:17.432140 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:17.432099    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"]
Oct 31 21:42:17.433845 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:17.433801    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:42:17.433979 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:17.433893    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:42:17.433979 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:17.433962    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:42:17.976525 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:17.976489    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:18.375704 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:18.375658    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk" event={"ID":"928636a0-a99a-43da-b29f-6ad518f022e1","Type":"ContainerStarted","Data":"b6242167b078c6ed8d290d52e301b250ee50f218aa5dc91fedb6634155510afc"}
Oct 31 21:42:18.376137 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:18.375732    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk" event={"ID":"928636a0-a99a-43da-b29f-6ad518f022e1","Type":"ContainerStarted","Data":"88b4f89f5938cbf3d989cdeab8cf62f3f8b4137623de8ad19deb9c56b46d4180"}
Oct 31 21:42:18.455452 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:18.455396    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk" podStartSLOduration=2.455382045 podStartE2EDuration="2.455382045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:42:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:42:18.43809137 +0000 UTC m=+1578.354400800" watchObservedRunningTime="2026-10-31 21:42:18.455382045 +0000 UTC m=+1578.371691475"
Oct 31 21:42:19.053459 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:19.053423    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:42:19.059378 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:19.059353    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:42:19.382679 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:19.382599    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:42:19.383657 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:19.383636    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:42:19.722457 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:19.722433    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:42:19.764855 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:19.764820    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:19.884967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:19.881116    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istio-data\") pod \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") "
Oct 31 21:42:19.884967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:19.881239    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-workload-socket\") pod \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") "
Oct 31 21:42:19.884967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:19.881280    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-kube-api-access-5kmqx\" (UniqueName: \"kubernetes.io/projected/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-kube-api-access-5kmqx\") pod \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") "
Oct 31 21:42:19.884967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:19.881307    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istio-envoy\") pod \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") "
Oct 31 21:42:19.884967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:19.881422    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-workload-certs\") pod \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") "
Oct 31 21:42:19.884967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:19.881531    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istio-podinfo\") pod \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") "
Oct 31 21:42:19.884967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:19.881636    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istio-token\" (UniqueName: \"kubernetes.io/projected/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istio-token\") pod \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") "
Oct 31 21:42:19.884967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:19.881691    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-credential-socket\") pod \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") "
Oct 31 21:42:19.884967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:19.881789    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istiod-ca-cert\") pod \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\" (UID: \"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041\") "
Oct 31 21:42:19.884967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:19.883104    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istiod-ca-cert" pod "d420b3f8-f5f8-4bd3-86b9-c4b6b6911041" (UID: "d420b3f8-f5f8-4bd3-86b9-c4b6b6911041"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:42:19.884967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:19.883400    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istio-data" pod "d420b3f8-f5f8-4bd3-86b9-c4b6b6911041" (UID: "d420b3f8-f5f8-4bd3-86b9-c4b6b6911041"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:42:19.884967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:19.883562    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-workload-socket" pod "d420b3f8-f5f8-4bd3-86b9-c4b6b6911041" (UID: "d420b3f8-f5f8-4bd3-86b9-c4b6b6911041"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:42:19.904268 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:19.895628    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-workload-certs" pod "d420b3f8-f5f8-4bd3-86b9-c4b6b6911041" (UID: "d420b3f8-f5f8-4bd3-86b9-c4b6b6911041"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:42:19.904268 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:19.895826    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-credential-socket" pod "d420b3f8-f5f8-4bd3-86b9-c4b6b6911041" (UID: "d420b3f8-f5f8-4bd3-86b9-c4b6b6911041"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:42:19.911791 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:19.911747    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istio-podinfo" pod "d420b3f8-f5f8-4bd3-86b9-c4b6b6911041" (UID: "d420b3f8-f5f8-4bd3-86b9-c4b6b6911041"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:42:19.913005 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:19.912973    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istio-token" pod "d420b3f8-f5f8-4bd3-86b9-c4b6b6911041" (UID: "d420b3f8-f5f8-4bd3-86b9-c4b6b6911041"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:42:19.913170 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:19.913144    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-kube-api-access-5kmqx" pod "d420b3f8-f5f8-4bd3-86b9-c4b6b6911041" (UID: "d420b3f8-f5f8-4bd3-86b9-c4b6b6911041"). InnerVolumeSpecName "kube-api-access-5kmqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:42:19.913242 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:19.913153    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istio-envoy" pod "d420b3f8-f5f8-4bd3-86b9-c4b6b6911041" (UID: "d420b3f8-f5f8-4bd3-86b9-c4b6b6911041"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:42:19.985621 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:19.984700    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:42:19.985621 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:19.984739    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:42:19.985621 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:19.984755    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:42:19.985621 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:19.984772    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:42:19.985621 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:19.984792    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:42:19.985621 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:19.984806    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:42:19.985621 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:19.984820    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:42:19.985621 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:19.984834    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5kmqx\" (UniqueName: \"kubernetes.io/projected/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-kube-api-access-5kmqx\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:42:19.985621 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:19.984852    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:42:20.391973 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:20.391926    4460 generic.go:356] "Generic (PLEG): container finished" podID="d420b3f8-f5f8-4bd3-86b9-c4b6b6911041" containerID="b5396875d149dbf3ccc4a5d3e536abc90adea3c634dd53f17908910da3a26364" exitCode=0
Oct 31 21:42:20.392410 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:20.391999    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"
Oct 31 21:42:20.392410 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:20.392018    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k" event={"ID":"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041","Type":"ContainerDied","Data":"b5396875d149dbf3ccc4a5d3e536abc90adea3c634dd53f17908910da3a26364"}
Oct 31 21:42:20.392410 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:20.392077    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k" event={"ID":"d420b3f8-f5f8-4bd3-86b9-c4b6b6911041","Type":"ContainerDied","Data":"59a98ce67aa2b6c8c2120ebe9e975f4cabd326468415d1d1e5e3881a903f1efb"}
Oct 31 21:42:20.392410 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:20.392102    4460 scope.go:122] "RemoveContainer" containerID="b5396875d149dbf3ccc4a5d3e536abc90adea3c634dd53f17908910da3a26364"
Oct 31 21:42:20.402482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:20.402453    4460 scope.go:122] "RemoveContainer" containerID="922bd049f4fd2e77e33921b23e8f2c87f6a090e9cea4e4dc185aa9a6ce105320"
Oct 31 21:42:20.411761 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:20.411741    4460 scope.go:122] "RemoveContainer" containerID="b5396875d149dbf3ccc4a5d3e536abc90adea3c634dd53f17908910da3a26364"
Oct 31 21:42:20.412072 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:42:20.412051    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5396875d149dbf3ccc4a5d3e536abc90adea3c634dd53f17908910da3a26364\": container with ID starting with b5396875d149dbf3ccc4a5d3e536abc90adea3c634dd53f17908910da3a26364 not found: ID does not exist" containerID="b5396875d149dbf3ccc4a5d3e536abc90adea3c634dd53f17908910da3a26364"
Oct 31 21:42:20.412126 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:20.412080    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5396875d149dbf3ccc4a5d3e536abc90adea3c634dd53f17908910da3a26364"} err="failed to get container status \"b5396875d149dbf3ccc4a5d3e536abc90adea3c634dd53f17908910da3a26364\": rpc error: code = NotFound desc = could not find container \"b5396875d149dbf3ccc4a5d3e536abc90adea3c634dd53f17908910da3a26364\": container with ID starting with b5396875d149dbf3ccc4a5d3e536abc90adea3c634dd53f17908910da3a26364 not found: ID does not exist"
Oct 31 21:42:20.412126 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:20.412097    4460 scope.go:122] "RemoveContainer" containerID="922bd049f4fd2e77e33921b23e8f2c87f6a090e9cea4e4dc185aa9a6ce105320"
Oct 31 21:42:20.412381 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:42:20.412362    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"922bd049f4fd2e77e33921b23e8f2c87f6a090e9cea4e4dc185aa9a6ce105320\": container with ID starting with 922bd049f4fd2e77e33921b23e8f2c87f6a090e9cea4e4dc185aa9a6ce105320 not found: ID does not exist" containerID="922bd049f4fd2e77e33921b23e8f2c87f6a090e9cea4e4dc185aa9a6ce105320"
Oct 31 21:42:20.412429 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:20.412387    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"922bd049f4fd2e77e33921b23e8f2c87f6a090e9cea4e4dc185aa9a6ce105320"} err="failed to get container status \"922bd049f4fd2e77e33921b23e8f2c87f6a090e9cea4e4dc185aa9a6ce105320\": rpc error: code = NotFound desc = could not find container \"922bd049f4fd2e77e33921b23e8f2c87f6a090e9cea4e4dc185aa9a6ce105320\": container with ID starting with 922bd049f4fd2e77e33921b23e8f2c87f6a090e9cea4e4dc185aa9a6ce105320 not found: ID does not exist"
Oct 31 21:42:20.450840 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:20.450807    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"]
Oct 31 21:42:20.464206 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:20.464171    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-9bbfc5-openshift-default-6bc8b6556c-qbt5k"]
Oct 31 21:42:20.838346 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:20.838298    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:20.950381 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:20.950338    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:21.453305 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:42:21.453271    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:42:21.756139 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:21.756038    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:21.864174 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:21.864135    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:21.978756 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:21.975612    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:22.226338 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:22.226304    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d420b3f8-f5f8-4bd3-86b9-c4b6b6911041" path="/var/lib/kubelet/pods/d420b3f8-f5f8-4bd3-86b9-c4b6b6911041/volumes"
Oct 31 21:42:22.768506 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:22.768469    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:22.880503 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:22.880461    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:22.931659 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:22.931625    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-80542b-777f94c5f9-p4p27"]
Oct 31 21:42:22.932308 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:22.932287    4460 cpu_manager.go:414] "containerMap: removing container" podUID="d420b3f8-f5f8-4bd3-86b9-c4b6b6911041" containerName="istio-proxy"
Oct 31 21:42:22.932308 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:22.932306    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d420b3f8-f5f8-4bd3-86b9-c4b6b6911041" containerName="istio-proxy"
Oct 31 21:42:22.932507 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:22.932329    4460 cpu_manager.go:414] "containerMap: removing container" podUID="d420b3f8-f5f8-4bd3-86b9-c4b6b6911041" containerName="istio-proxy"
Oct 31 21:42:22.932507 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:22.932337    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d420b3f8-f5f8-4bd3-86b9-c4b6b6911041" containerName="istio-proxy"
Oct 31 21:42:22.932507 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:22.932474    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="d420b3f8-f5f8-4bd3-86b9-c4b6b6911041" containerName="istio-proxy"
Oct 31 21:42:22.932507 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:22.932486    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="d420b3f8-f5f8-4bd3-86b9-c4b6b6911041" containerName="istio-proxy"
Oct 31 21:42:22.935834 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:22.935814    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-80542b-777f94c5f9-p4p27"
Oct 31 21:42:22.955045 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:22.955018    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-80542b-777f94c5f9-p4p27"]
Oct 31 21:42:22.964646 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:22.964609    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-a-80542b\""
Oct 31 21:42:22.964808 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:22.964708    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-a-80542b-dockercfg-jhhcz\""
Oct 31 21:42:23.050591 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:23.050485    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39007e3f-5368-4ae6-8d73-2e4d407cf76b-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-80542b-777f94c5f9-p4p27\" (UID: \"39007e3f-5368-4ae6-8d73-2e4d407cf76b\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-80542b-777f94c5f9-p4p27"
Oct 31 21:42:23.050778 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:23.050667    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntgmx\" (UniqueName: \"kubernetes.io/projected/39007e3f-5368-4ae6-8d73-2e4d407cf76b-kube-api-access-ntgmx\") pod \"payload-pre-processing-e2e-praxis-a-80542b-777f94c5f9-p4p27\" (UID: \"39007e3f-5368-4ae6-8d73-2e4d407cf76b\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-80542b-777f94c5f9-p4p27"
Oct 31 21:42:23.076963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:23.075502    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-80542b-6c5b644b9-v8vqz"]
Oct 31 21:42:23.216014 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:23.215151    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39007e3f-5368-4ae6-8d73-2e4d407cf76b-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-80542b-777f94c5f9-p4p27\" (UID: \"39007e3f-5368-4ae6-8d73-2e4d407cf76b\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-80542b-777f94c5f9-p4p27"
Oct 31 21:42:23.216014 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:23.215327    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntgmx\" (UniqueName: \"kubernetes.io/projected/39007e3f-5368-4ae6-8d73-2e4d407cf76b-kube-api-access-ntgmx\") pod \"payload-pre-processing-e2e-praxis-a-80542b-777f94c5f9-p4p27\" (UID: \"39007e3f-5368-4ae6-8d73-2e4d407cf76b\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-80542b-777f94c5f9-p4p27"
Oct 31 21:42:23.234037 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:23.233997    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39007e3f-5368-4ae6-8d73-2e4d407cf76b-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-80542b-777f94c5f9-p4p27\" (UID: \"39007e3f-5368-4ae6-8d73-2e4d407cf76b\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-80542b-777f94c5f9-p4p27"
Oct 31 21:42:23.300352 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:23.300313    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntgmx\" (UniqueName: \"kubernetes.io/projected/39007e3f-5368-4ae6-8d73-2e4d407cf76b-kube-api-access-ntgmx\") pod \"payload-pre-processing-e2e-praxis-a-80542b-777f94c5f9-p4p27\" (UID: \"39007e3f-5368-4ae6-8d73-2e4d407cf76b\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-80542b-777f94c5f9-p4p27"
Oct 31 21:42:23.341012 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:23.337378    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-80542b-6c5b644b9-v8vqz"
Oct 31 21:42:23.342958 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:23.342903    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-80542b-6c5b644b9-v8vqz"]
Oct 31 21:42:23.446234 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:23.432283    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6868309a-8813-4420-bc7d-4a5b7a38eb46-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-80542b-6c5b644b9-v8vqz\" (UID: \"6868309a-8813-4420-bc7d-4a5b7a38eb46\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-80542b-6c5b644b9-v8vqz"
Oct 31 21:42:23.446483 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:23.446442    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2ml2\" (UniqueName: \"kubernetes.io/projected/6868309a-8813-4420-bc7d-4a5b7a38eb46-kube-api-access-n2ml2\") pod \"payload-processing-e2e-praxis-a-80542b-6c5b644b9-v8vqz\" (UID: \"6868309a-8813-4420-bc7d-4a5b7a38eb46\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-80542b-6c5b644b9-v8vqz"
Oct 31 21:42:23.547890 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:23.547848    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6868309a-8813-4420-bc7d-4a5b7a38eb46-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-80542b-6c5b644b9-v8vqz\" (UID: \"6868309a-8813-4420-bc7d-4a5b7a38eb46\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-80542b-6c5b644b9-v8vqz"
Oct 31 21:42:23.547890 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:23.547895    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2ml2\" (UniqueName: \"kubernetes.io/projected/6868309a-8813-4420-bc7d-4a5b7a38eb46-kube-api-access-n2ml2\") pod \"payload-processing-e2e-praxis-a-80542b-6c5b644b9-v8vqz\" (UID: \"6868309a-8813-4420-bc7d-4a5b7a38eb46\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-80542b-6c5b644b9-v8vqz"
Oct 31 21:42:23.548701 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:23.548680    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6868309a-8813-4420-bc7d-4a5b7a38eb46-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-80542b-6c5b644b9-v8vqz\" (UID: \"6868309a-8813-4420-bc7d-4a5b7a38eb46\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-80542b-6c5b644b9-v8vqz"
Oct 31 21:42:23.551449 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:23.551430    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-80542b-777f94c5f9-p4p27"
Oct 31 21:42:23.574911 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:23.574882    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2ml2\" (UniqueName: \"kubernetes.io/projected/6868309a-8813-4420-bc7d-4a5b7a38eb46-kube-api-access-n2ml2\") pod \"payload-processing-e2e-praxis-a-80542b-6c5b644b9-v8vqz\" (UID: \"6868309a-8813-4420-bc7d-4a5b7a38eb46\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-80542b-6c5b644b9-v8vqz"
Oct 31 21:42:23.623484 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:23.623444    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:23.697471 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:23.685814    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-80542b-6c5b644b9-v8vqz"
Oct 31 21:42:23.868213 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:23.868183    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-80542b-777f94c5f9-p4p27"]
Oct 31 21:42:23.869773 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:42:23.869742    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39007e3f_5368_4ae6_8d73_2e4d407cf76b.slice/crio-7e35c6209482de0adf4a7764ca1a109639b98ee1b6c887e2eb9ba84e3a12c794 WatchSource:0}: Error finding container 7e35c6209482de0adf4a7764ca1a109639b98ee1b6c887e2eb9ba84e3a12c794: Status 404 returned error can't find the container with id 7e35c6209482de0adf4a7764ca1a109639b98ee1b6c887e2eb9ba84e3a12c794
Oct 31 21:42:24.069661 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:24.069630    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-80542b-6c5b644b9-v8vqz"]
Oct 31 21:42:24.070431 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:42:24.070408    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6868309a_8813_4420_bc7d_4a5b7a38eb46.slice/crio-479e392d63b536e77642dc78329de67c97abc0829840308a11b513ce9c09ff6a WatchSource:0}: Error finding container 479e392d63b536e77642dc78329de67c97abc0829840308a11b513ce9c09ff6a: Status 404 returned error can't find the container with id 479e392d63b536e77642dc78329de67c97abc0829840308a11b513ce9c09ff6a
Oct 31 21:42:24.427530 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:24.427422    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-80542b-777f94c5f9-p4p27" event={"ID":"39007e3f-5368-4ae6-8d73-2e4d407cf76b","Type":"ContainerStarted","Data":"f7baeb0b935c7f80a1c418bb81b2b79ff14069fae064004451712dc26bf6ba24"}
Oct 31 21:42:24.427530 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:24.427490    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-80542b-777f94c5f9-p4p27"
Oct 31 21:42:24.427530 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:24.427512    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-80542b-777f94c5f9-p4p27" event={"ID":"39007e3f-5368-4ae6-8d73-2e4d407cf76b","Type":"ContainerStarted","Data":"7e35c6209482de0adf4a7764ca1a109639b98ee1b6c887e2eb9ba84e3a12c794"}
Oct 31 21:42:24.429278 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:24.429243    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-80542b-6c5b644b9-v8vqz" event={"ID":"6868309a-8813-4420-bc7d-4a5b7a38eb46","Type":"ContainerStarted","Data":"86558919e547044d38698f268e7b27e614a6089b58638b8067e36b00dfa84620"}
Oct 31 21:42:24.429414 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:24.429301    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-80542b-6c5b644b9-v8vqz" event={"ID":"6868309a-8813-4420-bc7d-4a5b7a38eb46","Type":"ContainerStarted","Data":"479e392d63b536e77642dc78329de67c97abc0829840308a11b513ce9c09ff6a"}
Oct 31 21:42:24.429414 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:24.429329    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-80542b-6c5b644b9-v8vqz"
Oct 31 21:42:24.473701 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:24.473641    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-80542b-777f94c5f9-p4p27" podStartSLOduration=2.473626259 podStartE2EDuration="2.473626259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:42:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:42:24.471225395 +0000 UTC m=+1584.387534824" watchObservedRunningTime="2026-10-31 21:42:24.473626259 +0000 UTC m=+1584.389935688"
Oct 31 21:42:24.515845 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:24.515790    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-praxis-a-80542b-6c5b644b9-v8vqz" podStartSLOduration=1.515775245 podStartE2EDuration="1.515775245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:42:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:42:24.514766948 +0000 UTC m=+1584.431076378" watchObservedRunningTime="2026-10-31 21:42:24.515775245 +0000 UTC m=+1584.432084714"
Oct 31 21:42:25.339909 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:25.339872    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:26.968443 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:26.968387    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:27.078792 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:27.078757    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:27.189137 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:27.189104    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:28.722534 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:28.722488    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:30.415258 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:30.415224    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:32.117966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:32.112086    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:33.832491 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:33.832455    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:35.249630 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:35.249591    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:35.433413 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:35.433378    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:35.466099 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:35.466067    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-80542b-6c5b644b9-v8vqz"
Oct 31 21:42:35.466272 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:35.466141    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-80542b-777f94c5f9-p4p27"
Oct 31 21:42:35.545456 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:35.545424    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:35.834201 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:35.834106    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:35.963145 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:35.963117    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:36.108885 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:36.108803    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:36.230305 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:36.230247    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:36.492451 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:36.492373    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:37.428062 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:37.428029    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:37.893387 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:37.893351    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:38.266854 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:38.266819    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:38.425170 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:38.425127    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:40.116759 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:40.116724    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:41.420776 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:41.420667    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:41.482908 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:41.482870    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:42:41.494182 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:41.494155    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9"]
Oct 31 21:42:41.546037 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:41.545999    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:41.678854 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:41.678770    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:42.137224 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:42.137144    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:42.251255 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:42.251216    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:43.709748 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:43.709507    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:43.903152 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:43.903117    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:44.176965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:44.025780    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-bb9990-644f4468f9-tvk5r"]
Oct 31 21:42:44.183514 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:44.062494    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:44.470036 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:44.469947    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-bb9990-644f4468f9-tvk5r"]
Oct 31 21:42:44.471312 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:44.471286    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-bb9990-644f4468f9-tvk5r"
Oct 31 21:42:44.517809 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:44.514498    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-b-bb9990\""
Oct 31 21:42:44.517809 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:44.514611    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-b-bb9990-dockercfg-gdl99\""
Oct 31 21:42:44.607081 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:44.606610    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dc613a9-a4d1-437d-87df-9c13daa4bb05-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-bb9990-644f4468f9-tvk5r\" (UID: \"4dc613a9-a4d1-437d-87df-9c13daa4bb05\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-bb9990-644f4468f9-tvk5r"
Oct 31 21:42:44.607081 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:44.606821    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqp5k\" (UniqueName: \"kubernetes.io/projected/4dc613a9-a4d1-437d-87df-9c13daa4bb05-kube-api-access-qqp5k\") pod \"payload-processing-e2e-praxis-b-bb9990-644f4468f9-tvk5r\" (UID: \"4dc613a9-a4d1-437d-87df-9c13daa4bb05\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-bb9990-644f4468f9-tvk5r"
Oct 31 21:42:44.708655 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:44.708614    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqp5k\" (UniqueName: \"kubernetes.io/projected/4dc613a9-a4d1-437d-87df-9c13daa4bb05-kube-api-access-qqp5k\") pod \"payload-processing-e2e-praxis-b-bb9990-644f4468f9-tvk5r\" (UID: \"4dc613a9-a4d1-437d-87df-9c13daa4bb05\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-bb9990-644f4468f9-tvk5r"
Oct 31 21:42:44.708824 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:44.708805    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dc613a9-a4d1-437d-87df-9c13daa4bb05-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-bb9990-644f4468f9-tvk5r\" (UID: \"4dc613a9-a4d1-437d-87df-9c13daa4bb05\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-bb9990-644f4468f9-tvk5r"
Oct 31 21:42:44.709728 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:44.709706    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dc613a9-a4d1-437d-87df-9c13daa4bb05-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-bb9990-644f4468f9-tvk5r\" (UID: \"4dc613a9-a4d1-437d-87df-9c13daa4bb05\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-bb9990-644f4468f9-tvk5r"
Oct 31 21:42:44.772022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:44.758439    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqp5k\" (UniqueName: \"kubernetes.io/projected/4dc613a9-a4d1-437d-87df-9c13daa4bb05-kube-api-access-qqp5k\") pod \"payload-processing-e2e-praxis-b-bb9990-644f4468f9-tvk5r\" (UID: \"4dc613a9-a4d1-437d-87df-9c13daa4bb05\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-bb9990-644f4468f9-tvk5r"
Oct 31 21:42:44.915103 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:44.915064    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-bb9990-644f4468f9-tvk5r"
Oct 31 21:42:45.707993 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:45.707956    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:45.719316 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:45.719285    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-bb9990-644f4468f9-tvk5r"]
Oct 31 21:42:46.744981 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:46.744920    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:46.791018 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:46.785263    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-bb9990-644f4468f9-tvk5r" event={"ID":"4dc613a9-a4d1-437d-87df-9c13daa4bb05","Type":"ContainerStarted","Data":"bf601a0b86d6909c149a640bb5b23bfccb34404418aa9e9221b8187da83b981b"}
Oct 31 21:42:46.791018 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:46.785343    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-bb9990-644f4468f9-tvk5r" event={"ID":"4dc613a9-a4d1-437d-87df-9c13daa4bb05","Type":"ContainerStarted","Data":"2c536ea0e1a111ebc7b0941054c32d6c722889351b7f2dca0606604ac1b0a4fb"}
Oct 31 21:42:46.795885 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:46.795802    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-praxis-b-bb9990-644f4468f9-tvk5r"
Oct 31 21:42:46.874974 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:46.874569    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-praxis-b-bb9990-644f4468f9-tvk5r" podStartSLOduration=3.874547248 podStartE2EDuration="3.874547248s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:42:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:42:46.864255122 +0000 UTC m=+1606.780564554" watchObservedRunningTime="2026-10-31 21:42:46.874547248 +0000 UTC m=+1606.790856659"
Oct 31 21:42:47.396972 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:47.395671    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:49.150962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:49.150886    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:49.673515 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:49.673421    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:49.784256 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:49.784221    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:50.822524 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:50.822440    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:51.287099 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:51.287049    4460 patch_prober.go:29] interesting pod/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 21:42:51.287296 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:51.287114    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9" podUID="3788ae03-d501-4b1f-89e2-c47c533ead04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:42:51.465093 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:42:51.465023    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:42:52.511999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:52.511912    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:54.370574 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:54.370489    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:54.485409 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:54.485335    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:56.013522 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:56.013486    4460 generic.go:356] "Generic (PLEG): container finished" podID="3788ae03-d501-4b1f-89e2-c47c533ead04" containerID="7dbb1b345324e0163f7ab4bc458a9797dae160c24aa274230203fa861d6b0197" exitCode=0
Oct 31 21:42:56.014221 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:56.014195    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9" event={"ID":"3788ae03-d501-4b1f-89e2-c47c533ead04","Type":"ContainerDied","Data":"7dbb1b345324e0163f7ab4bc458a9797dae160c24aa274230203fa861d6b0197"}
Oct 31 21:42:56.014865 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:56.014846    4460 scope.go:122] "RemoveContainer" containerID="7dbb1b345324e0163f7ab4bc458a9797dae160c24aa274230203fa861d6b0197"
Oct 31 21:42:56.023046 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:56.023014    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:42:56.023283 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:56.023257    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:42:56.023446 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:56.023429    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:42:56.387985 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:56.387930    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:56.516150 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:56.516110    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9" podUID="e965141b-d468-4dd9-80a0-018b9af1cfc0" containerName="main" containerID="cri-o://dc87eafd3a96beb66e31cef528eaeb09720556834e2b54d8019c46cd483a1e86" gracePeriod=45
Oct 31 21:42:56.793125 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:56.791334    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9" podUID="e965141b-d468-4dd9-80a0-018b9af1cfc0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.114:8000/ready\": dial tcp 10.133.0.114:8000: connect: connection refused"
Oct 31 21:42:57.058759 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.058689    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9"
Oct 31 21:42:57.059214 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.058920    4460 generic.go:356] "Generic (PLEG): container finished" podID="2b619644-e691-44de-889f-782d25668bca" containerID="ffe58be85d96ec3e16e38eb96c5ea214b280066966d7ebff1efe09da52ed886c" exitCode=0
Oct 31 21:42:57.059214 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.058987    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648" event={"ID":"2b619644-e691-44de-889f-782d25668bca","Type":"ContainerDied","Data":"ffe58be85d96ec3e16e38eb96c5ea214b280066966d7ebff1efe09da52ed886c"}
Oct 31 21:42:57.059500 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.059482    4460 scope.go:122] "RemoveContainer" containerID="ffe58be85d96ec3e16e38eb96c5ea214b280066966d7ebff1efe09da52ed886c"
Oct 31 21:42:57.061041 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.061021    4460 generic.go:356] "Generic (PLEG): container finished" podID="1f5fd98d-9122-4102-b317-d51242d31d0e" containerID="d0afd41ea2de72627ed621723c6cc15a3e378594eb25ebfe0cbe61d6cffaee12" exitCode=0
Oct 31 21:42:57.061119 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.061088    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj" event={"ID":"1f5fd98d-9122-4102-b317-d51242d31d0e","Type":"ContainerDied","Data":"d0afd41ea2de72627ed621723c6cc15a3e378594eb25ebfe0cbe61d6cffaee12"}
Oct 31 21:42:57.061216 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.061191    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:42:57.061267 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.061255    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:42:57.061303 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.061289    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:42:57.061423 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.061410    4460 scope.go:122] "RemoveContainer" containerID="d0afd41ea2de72627ed621723c6cc15a3e378594eb25ebfe0cbe61d6cffaee12"
Oct 31 21:42:57.063016 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.062987    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:42:57.063095 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.063066    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:42:57.063143 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.063101    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:42:57.063633 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.063613    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9" event={"ID":"3788ae03-d501-4b1f-89e2-c47c533ead04","Type":"ContainerStarted","Data":"2300bad6f458d416ac41c8241319d446675d7bae06d11947436c715898284d6c"}
Oct 31 21:42:57.064895 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.064875    4460 generic.go:356] "Generic (PLEG): container finished" podID="e965141b-d468-4dd9-80a0-018b9af1cfc0" containerID="dc87eafd3a96beb66e31cef528eaeb09720556834e2b54d8019c46cd483a1e86" exitCode=0
Oct 31 21:42:57.065027 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.064983    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9" event={"ID":"e965141b-d468-4dd9-80a0-018b9af1cfc0","Type":"ContainerDied","Data":"dc87eafd3a96beb66e31cef528eaeb09720556834e2b54d8019c46cd483a1e86"}
Oct 31 21:42:57.065027 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.065012    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9"
Oct 31 21:42:57.065155 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.065012    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9" event={"ID":"e965141b-d468-4dd9-80a0-018b9af1cfc0","Type":"ContainerDied","Data":"2a55348fe85dd08dc2f71a41520c9247ca45146c5da218ee09fe93942c583063"}
Oct 31 21:42:57.065155 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.065071    4460 scope.go:122] "RemoveContainer" containerID="dc87eafd3a96beb66e31cef528eaeb09720556834e2b54d8019c46cd483a1e86"
Oct 31 21:42:57.077424 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.077398    4460 scope.go:122] "RemoveContainer" containerID="dc87eafd3a96beb66e31cef528eaeb09720556834e2b54d8019c46cd483a1e86"
Oct 31 21:42:57.077716 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:42:57.077691    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc87eafd3a96beb66e31cef528eaeb09720556834e2b54d8019c46cd483a1e86\": container with ID starting with dc87eafd3a96beb66e31cef528eaeb09720556834e2b54d8019c46cd483a1e86 not found: ID does not exist" containerID="dc87eafd3a96beb66e31cef528eaeb09720556834e2b54d8019c46cd483a1e86"
Oct 31 21:42:57.077808 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.077727    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc87eafd3a96beb66e31cef528eaeb09720556834e2b54d8019c46cd483a1e86"} err="failed to get container status \"dc87eafd3a96beb66e31cef528eaeb09720556834e2b54d8019c46cd483a1e86\": rpc error: code = NotFound desc = could not find container \"dc87eafd3a96beb66e31cef528eaeb09720556834e2b54d8019c46cd483a1e86\": container with ID starting with dc87eafd3a96beb66e31cef528eaeb09720556834e2b54d8019c46cd483a1e86 not found: ID does not exist"
Oct 31 21:42:57.089652 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.089632    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:42:57.089838 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.089817    4460 patch_prober.go:29] interesting pod/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused" start-of-body=
Oct 31 21:42:57.089903 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.089876    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9" podUID="3788ae03-d501-4b1f-89e2-c47c533ead04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused"
Oct 31 21:42:57.148192 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.138086    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e965141b-d468-4dd9-80a0-018b9af1cfc0-tls-certs\" (UniqueName: \"kubernetes.io/secret/e965141b-d468-4dd9-80a0-018b9af1cfc0-tls-certs\") pod \"e965141b-d468-4dd9-80a0-018b9af1cfc0\" (UID: \"e965141b-d468-4dd9-80a0-018b9af1cfc0\") "
Oct 31 21:42:57.148192 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.138329    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e965141b-d468-4dd9-80a0-018b9af1cfc0-kube-api-access-hnxtj\" (UniqueName: \"kubernetes.io/projected/e965141b-d468-4dd9-80a0-018b9af1cfc0-kube-api-access-hnxtj\") pod \"e965141b-d468-4dd9-80a0-018b9af1cfc0\" (UID: \"e965141b-d468-4dd9-80a0-018b9af1cfc0\") "
Oct 31 21:42:57.157963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.151243    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e965141b-d468-4dd9-80a0-018b9af1cfc0-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e965141b-d468-4dd9-80a0-018b9af1cfc0-model-cache\") pod \"e965141b-d468-4dd9-80a0-018b9af1cfc0\" (UID: \"e965141b-d468-4dd9-80a0-018b9af1cfc0\") "
Oct 31 21:42:57.157963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.151403    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e965141b-d468-4dd9-80a0-018b9af1cfc0-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e965141b-d468-4dd9-80a0-018b9af1cfc0-dshm\") pod \"e965141b-d468-4dd9-80a0-018b9af1cfc0\" (UID: \"e965141b-d468-4dd9-80a0-018b9af1cfc0\") "
Oct 31 21:42:57.157963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.151499    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e965141b-d468-4dd9-80a0-018b9af1cfc0-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e965141b-d468-4dd9-80a0-018b9af1cfc0-tmp-dir\") pod \"e965141b-d468-4dd9-80a0-018b9af1cfc0\" (UID: \"e965141b-d468-4dd9-80a0-018b9af1cfc0\") "
Oct 31 21:42:57.157963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.151719    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e965141b-d468-4dd9-80a0-018b9af1cfc0-home\" (UniqueName: \"kubernetes.io/empty-dir/e965141b-d468-4dd9-80a0-018b9af1cfc0-home\") pod \"e965141b-d468-4dd9-80a0-018b9af1cfc0\" (UID: \"e965141b-d468-4dd9-80a0-018b9af1cfc0\") "
Oct 31 21:42:57.157963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.152746    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e965141b-d468-4dd9-80a0-018b9af1cfc0-home" pod "e965141b-d468-4dd9-80a0-018b9af1cfc0" (UID: "e965141b-d468-4dd9-80a0-018b9af1cfc0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:42:57.157963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.153047    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e965141b-d468-4dd9-80a0-018b9af1cfc0-model-cache" pod "e965141b-d468-4dd9-80a0-018b9af1cfc0" (UID: "e965141b-d468-4dd9-80a0-018b9af1cfc0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:42:57.157963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.154004    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e965141b-d468-4dd9-80a0-018b9af1cfc0-tls-certs" pod "e965141b-d468-4dd9-80a0-018b9af1cfc0" (UID: "e965141b-d468-4dd9-80a0-018b9af1cfc0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 21:42:57.157963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.154111    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e965141b-d468-4dd9-80a0-018b9af1cfc0-tmp-dir" pod "e965141b-d468-4dd9-80a0-018b9af1cfc0" (UID: "e965141b-d468-4dd9-80a0-018b9af1cfc0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:42:57.157963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.155205    4460 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e965141b-d468-4dd9-80a0-018b9af1cfc0-home\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:42:57.157963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.155228    4460 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e965141b-d468-4dd9-80a0-018b9af1cfc0-model-cache\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:42:57.157963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.157121    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e965141b-d468-4dd9-80a0-018b9af1cfc0-kube-api-access-hnxtj" pod "e965141b-d468-4dd9-80a0-018b9af1cfc0" (UID: "e965141b-d468-4dd9-80a0-018b9af1cfc0"). InnerVolumeSpecName "kube-api-access-hnxtj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:42:57.158689 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.158337    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e965141b-d468-4dd9-80a0-018b9af1cfc0-dshm" pod "e965141b-d468-4dd9-80a0-018b9af1cfc0" (UID: "e965141b-d468-4dd9-80a0-018b9af1cfc0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:42:57.256928 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.256881    4460 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e965141b-d468-4dd9-80a0-018b9af1cfc0-tls-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:42:57.257111 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.256931    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnxtj\" (UniqueName: \"kubernetes.io/projected/e965141b-d468-4dd9-80a0-018b9af1cfc0-kube-api-access-hnxtj\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:42:57.257111 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.256964    4460 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e965141b-d468-4dd9-80a0-018b9af1cfc0-dshm\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:42:57.257111 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.257006    4460 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e965141b-d468-4dd9-80a0-018b9af1cfc0-tmp-dir\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:42:57.441359 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.441294    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9"]
Oct 31 21:42:57.465671 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.465622    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-fc0d65/ipp-route-fc0d65-kserve-dcf46dc4f-b22n9"]
Oct 31 21:42:57.514031 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:57.513988    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:58.092960 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:58.090548    4460 patch_prober.go:29] interesting pod/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused" start-of-body=
Oct 31 21:42:58.092960 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:58.090638    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9" podUID="3788ae03-d501-4b1f-89e2-c47c533ead04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused"
Oct 31 21:42:58.096996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:58.094851    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648" event={"ID":"2b619644-e691-44de-889f-782d25668bca","Type":"ContainerStarted","Data":"82562daaec4145194690b8821b6db73e660c70e6f2deb0a4d263ee3d8ca1e687"}
Oct 31 21:42:58.104967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:58.100751    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj" event={"ID":"1f5fd98d-9122-4102-b317-d51242d31d0e","Type":"ContainerStarted","Data":"13e50e9fcf11ba1cbb82650bac346c1f8a8780b1eef1b91771b802ef512720ad"}
Oct 31 21:42:58.120997 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:58.120840    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:58.304015 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:58.293537    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e965141b-d468-4dd9-80a0-018b9af1cfc0" path="/var/lib/kubelet/pods/e965141b-d468-4dd9-80a0-018b9af1cfc0/volumes"
Oct 31 21:42:58.343989 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:58.340079    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:42:58.343989 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:58.340204    4460 patch_prober.go:29] interesting pod/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 31 21:42:58.343989 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:58.340245    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj" podUID="1f5fd98d-9122-4102-b317-d51242d31d0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 31 21:42:58.736075 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:58.734133    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:58.858978 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:58.850121    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-praxis-b-bb9990-644f4468f9-tvk5r"
Oct 31 21:42:59.063059 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.063009    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:42:59.138116 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.078927    4460 patch_prober.go:29] interesting pod/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 31 21:42:59.138116 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.079012    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648" podUID="2b619644-e691-44de-889f-782d25668bca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 31 21:42:59.138719 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.138578    4460 patch_prober.go:29] interesting pod/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused" start-of-body=
Oct 31 21:42:59.138719 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.138635    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9" podUID="3788ae03-d501-4b1f-89e2-c47c533ead04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused"
Oct 31 21:42:59.296420 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.270023    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:59.296420 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.276100    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"]
Oct 31 21:42:59.296420 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.282402    4460 cpu_manager.go:414] "containerMap: removing container" podUID="e965141b-d468-4dd9-80a0-018b9af1cfc0" containerName="main"
Oct 31 21:42:59.296420 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.282427    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e965141b-d468-4dd9-80a0-018b9af1cfc0" containerName="main"
Oct 31 21:42:59.296420 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.282664    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="e965141b-d468-4dd9-80a0-018b9af1cfc0" containerName="main"
Oct 31 21:42:59.320457 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.319774    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:42:59.358968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.323794    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747" podUID="d1078aa7-1947-45ce-b6bb-d6550ad61fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:42:59.399299 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.336382    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istio-envoy\") pod \"e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") " pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:42:59.399299 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.390891    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-workload-socket\") pod \"e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") " pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:42:59.399299 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.390963    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-credential-socket\") pod \"e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") " pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:42:59.399299 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.391011    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istio-data\") pod \"e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") " pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:42:59.399299 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.391243    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istio-token\") pod \"e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") " pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:42:59.399299 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.391553    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istio-podinfo\") pod \"e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") " pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:42:59.399299 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.391737    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-workload-certs\") pod \"e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") " pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:42:59.399299 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.391804    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istiod-ca-cert\") pod \"e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") " pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:42:59.399299 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.391891    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r9cl\" (UniqueName: \"kubernetes.io/projected/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-kube-api-access-5r9cl\") pod \"e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") " pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:42:59.399299 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.337676    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-b-f4569e-openshift-default-dockercfg-5v84r\""
Oct 31 21:42:59.399299 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.364624    4460 patch_prober.go:29] interesting pod/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 31 21:42:59.437826 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.437787    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj" podUID="1f5fd98d-9122-4102-b317-d51242d31d0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 31 21:42:59.487090 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.467548    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:42:59.487479 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.487438    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh" podUID="869ed893-352c-4f8f-a174-b42afdeb4de7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:42:59.524581 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.496548    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istio-podinfo\") pod \"e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") " pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:42:59.608886 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.608801    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-workload-certs\") pod \"e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") " pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:42:59.609634 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.590668    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istio-podinfo\") pod \"e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") " pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:42:59.609792 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.524850    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"]
Oct 31 21:42:59.609952 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.609520    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-workload-certs\") pod \"e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") " pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:42:59.610119 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.610098    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istiod-ca-cert\") pod \"e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") " pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:42:59.661024 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.660994    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r9cl\" (UniqueName: \"kubernetes.io/projected/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-kube-api-access-5r9cl\") pod \"e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") " pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:42:59.661600 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.659751    4460 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n" podUID="4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:42:59.661825 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.660888    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istiod-ca-cert\") pod \"e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") " pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:42:59.661918 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.659638    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt" podUID="9771611f-879a-44f6-9c1a-f8647dc2d841" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 31 21:42:59.681095 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.681059    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istio-envoy\") pod \"e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") " pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:42:59.681809 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.681782    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-workload-socket\") pod \"e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") " pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:42:59.872025 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.748200    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-credential-socket\") pod \"e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") " pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:42:59.872025 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.748268    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istio-data\") pod \"e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") " pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:42:59.872025 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.748396    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istio-token\") pod \"e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") " pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:42:59.872025 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.689357    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-workload-socket\") pod \"e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") " pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:42:59.872025 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.749162    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-credential-socket\") pod \"e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") " pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:42:59.872025 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.741200    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" podUID="af7e0e0c-cf52-4270-b528-35718339996c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:42:59.872025 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.749404    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istio-data\") pod \"e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") " pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:42:59.872025 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.741094    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n" podUID="4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:42:59.872025 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.750427    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istio-envoy\") pod \"e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") " pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:42:59.872025 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.780474    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r9cl\" (UniqueName: \"kubernetes.io/projected/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-kube-api-access-5r9cl\") pod \"e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") " pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:42:59.936995 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.936337    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz" podUID="93a42128-ee74-4dcd-b158-27e33cfd44f9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:42:59.938979 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.937527    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podUID="9cd67bf5-8833-4a82-bc33-bed7bdfd4d36" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:42:59.980181 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.961312    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istio-token\") pod \"e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") " pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:42:59.980181 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:42:59.978360    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:00.021484 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:00.018834    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:43:00.030742 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:00.030461    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:43:00.160449 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:00.152413    4460 patch_prober.go:29] interesting pod/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 31 21:43:00.160449 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:00.152478    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648" podUID="2b619644-e691-44de-889f-782d25668bca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 31 21:43:00.191687 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:00.178867    4460 patch_prober.go:29] interesting pod/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused" start-of-body=
Oct 31 21:43:00.191687 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:00.178919    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9" podUID="3788ae03-d501-4b1f-89e2-c47c533ead04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused"
Oct 31 21:43:00.371969 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:00.358178    4460 patch_prober.go:29] interesting pod/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 31 21:43:00.371969 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:00.358241    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj" podUID="1f5fd98d-9122-4102-b317-d51242d31d0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 31 21:43:00.518441 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:00.492777    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747" podUID="d1078aa7-1947-45ce-b6bb-d6550ad61fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 31 21:43:00.604964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:00.571243    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh" podUID="869ed893-352c-4f8f-a174-b42afdeb4de7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:43:00.715405 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:00.713102    4460 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz" podUID="93a42128-ee74-4dcd-b158-27e33cfd44f9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:43:00.822137 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:00.814015    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n" podUID="4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": context deadline exceeded"
Oct 31 21:43:00.874114 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:00.874048    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" podUID="af7e0e0c-cf52-4270-b528-35718339996c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:43:01.118305 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:01.118261    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz" podUID="93a42128-ee74-4dcd-b158-27e33cfd44f9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded"
Oct 31 21:43:01.119593 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:01.119561    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podUID="9cd67bf5-8833-4a82-bc33-bed7bdfd4d36" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": context deadline exceeded"
Oct 31 21:43:01.219967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:01.214419    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 31 21:43:01.280571 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:01.269550    4460 patch_prober.go:29] interesting pod/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 31 21:43:01.280571 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:01.269608    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648" podUID="2b619644-e691-44de-889f-782d25668bca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 31 21:43:01.280571 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:01.275072    4460 patch_prober.go:29] interesting pod/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused" start-of-body=
Oct 31 21:43:01.280571 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:01.275122    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9" podUID="3788ae03-d501-4b1f-89e2-c47c533ead04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused"
Oct 31 21:43:01.358792 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:43:01.355102    4460 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.114s"
Oct 31 21:43:01.358792 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:01.355213    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:43:01.358792 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:01.355361    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 21:43:01.358792 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:01.355386    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"]
Oct 31 21:43:01.358792 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:01.355472    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:43:01.358792 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:01.355531    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:43:01.358792 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:01.355586    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:43:01.358792 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:01.355642    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:43:01.358792 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:01.355697    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:43:01.385677 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:01.385534    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:43:01.467993 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:01.464179    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:43:01.566439 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:01.566378    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:43:01.596964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:01.437064    4460 patch_prober.go:29] interesting pod/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 31 21:43:02.094968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:02.092287    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj" podUID="1f5fd98d-9122-4102-b317-d51242d31d0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 31 21:43:02.094968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:01.841285    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt" podUID="9771611f-879a-44f6-9c1a-f8647dc2d841" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 31 21:43:02.140075 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:02.118997    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh" podUID="869ed893-352c-4f8f-a174-b42afdeb4de7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.77:8000/ready\": context deadline exceeded"
Oct 31 21:43:02.140075 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:02.119595    4460 patch_prober.go:29] interesting pod/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 31 21:43:02.140075 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:02.119630    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648" podUID="2b619644-e691-44de-889f-782d25668bca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 31 21:43:02.145972 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:02.144030    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podUID="9cd67bf5-8833-4a82-bc33-bed7bdfd4d36" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:43:02.145972 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:02.144414    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" podUID="af7e0e0c-cf52-4270-b528-35718339996c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Oct 31 21:43:02.145972 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:02.144856    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz" podUID="93a42128-ee74-4dcd-b158-27e33cfd44f9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:43:02.145972 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:02.145572    4460 patch_prober.go:29] interesting pod/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused" start-of-body=
Oct 31 21:43:02.145972 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:02.145607    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9" podUID="3788ae03-d501-4b1f-89e2-c47c533ead04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused"
Oct 31 21:43:02.160825 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:01.520261    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747" podUID="d1078aa7-1947-45ce-b6bb-d6550ad61fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 31 21:43:02.200291 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:01.842751    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n" podUID="4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:43:02.233086 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:02.233034    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:43:02.302961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:02.288337    4460 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" podUID="af7e0e0c-cf52-4270-b528-35718339996c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:43:02.364062 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:02.309344    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:02.398876 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:02.398642    4460 patch_prober.go:29] interesting pod/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 31 21:43:02.398876 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:02.398719    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj" podUID="1f5fd98d-9122-4102-b317-d51242d31d0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 31 21:43:02.864059 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:02.836771    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:43:02.909356 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:02.909317    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc" event={"ID":"19fcad1c-5c6e-4274-936d-bf93ca76c7d4","Type":"ContainerStarted","Data":"02bf264760c542bb6e531057b914e3652b7497e7b3906b93f26c7ab363b7087e"}
Oct 31 21:43:03.123030 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:03.122987    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt" podUID="9771611f-879a-44f6-9c1a-f8647dc2d841" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 31 21:43:03.124015 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:03.120795    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh" podUID="869ed893-352c-4f8f-a174-b42afdeb4de7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:43:03.140012 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:03.139981    4460 patch_prober.go:29] interesting pod/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 31 21:43:03.140243 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:03.140207    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648" podUID="2b619644-e691-44de-889f-782d25668bca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 31 21:43:03.164199 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:03.164101    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747" podUID="d1078aa7-1947-45ce-b6bb-d6550ad61fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 31 21:43:03.166773 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:03.166752    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:43:03.184335 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:03.184296    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podUID="9cd67bf5-8833-4a82-bc33-bed7bdfd4d36" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": context deadline exceeded"
Oct 31 21:43:03.185599 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:03.185569    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" podUID="af7e0e0c-cf52-4270-b528-35718339996c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Oct 31 21:43:03.186351 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:03.186327    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz" podUID="93a42128-ee74-4dcd-b158-27e33cfd44f9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded"
Oct 31 21:43:03.215445 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:03.215393    4460 patch_prober.go:29] interesting pod/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused" start-of-body=
Oct 31 21:43:03.215724 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:03.215692    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9" podUID="3788ae03-d501-4b1f-89e2-c47c533ead04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused"
Oct 31 21:43:03.270053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:03.237538    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:43:03.270053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:03.262053    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n" podUID="4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:43:03.380962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:03.358162    4460 patch_prober.go:29] interesting pod/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 31 21:43:03.381317 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:03.381280    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj" podUID="1f5fd98d-9122-4102-b317-d51242d31d0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 31 21:43:03.716549 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:03.716459    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 21:43:03.719277 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:03.719251    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 21:43:04.012046 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:04.010518    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:04.119448 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:04.118317    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:43:04.130180 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:04.130139    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt" podUID="9771611f-879a-44f6-9c1a-f8647dc2d841" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:43:04.131582 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:04.131546    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh" podUID="869ed893-352c-4f8f-a174-b42afdeb4de7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:43:04.152649 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:04.134100    4460 patch_prober.go:29] interesting pod/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 31 21:43:04.152649 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:04.134146    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648" podUID="2b619644-e691-44de-889f-782d25668bca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 31 21:43:04.153832 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:04.153802    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:04.209964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:04.169153    4460 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh" podUID="869ed893-352c-4f8f-a174-b42afdeb4de7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.77:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:43:04.210313 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:04.210275    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podUID="9cd67bf5-8833-4a82-bc33-bed7bdfd4d36" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:43:04.210783 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:04.210757    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz" podUID="93a42128-ee74-4dcd-b158-27e33cfd44f9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:43:04.305777 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:04.305680    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:43:04.306699 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:04.306667    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n" podUID="4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:43:04.358365 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:04.358331    4460 patch_prober.go:29] interesting pod/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 31 21:43:04.358616 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:04.358588    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj" podUID="1f5fd98d-9122-4102-b317-d51242d31d0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 31 21:43:04.422966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:04.413694    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:43:04.425064 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:04.425042    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 21:43:04.490215 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:04.490187    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:43:04.506465 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:04.506425    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:43:04.729018 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:04.727901    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" podUID="af7e0e0c-cf52-4270-b528-35718339996c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:43:04.818765 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:04.818728    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 21:43:05.232963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:05.230140    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podUID="9cd67bf5-8833-4a82-bc33-bed7bdfd4d36" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:43:05.234196 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:05.234166    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt" podUID="9771611f-879a-44f6-9c1a-f8647dc2d841" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Oct 31 21:43:05.250321 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:05.250278    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh" podUID="869ed893-352c-4f8f-a174-b42afdeb4de7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.77:8000/ready\": context deadline exceeded"
Oct 31 21:43:05.273531 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:05.273494    4460 patch_prober.go:29] interesting pod/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 31 21:43:05.273767 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:05.273740    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648" podUID="2b619644-e691-44de-889f-782d25668bca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 31 21:43:05.277960 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:05.275114    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747" podUID="d1078aa7-1947-45ce-b6bb-d6550ad61fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:43:05.308150 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:05.308113    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:43:05.342398 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:05.342361    4460 patch_prober.go:29] interesting pod/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused" start-of-body=
Oct 31 21:43:05.342680 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:05.342638    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj" podUID="1f5fd98d-9122-4102-b317-d51242d31d0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.118:15021/healthz/ready\": dial tcp 10.133.0.118:15021: connect: connection refused"
Oct 31 21:43:05.418679 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:05.418646    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 21:43:05.481686 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:05.481642    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 21:43:05.491330 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:05.490588    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 21:43:05.496970 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:05.495754    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 21:43:05.567258 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:05.566836    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc" event={"ID":"19fcad1c-5c6e-4274-936d-bf93ca76c7d4","Type":"ContainerStarted","Data":"2107324f230fcb14ac11586aff5b1c9a7693440ded3798f4910f5296f1c59b06"}
Oct 31 21:43:05.663963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:05.663366    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:05.712362 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:05.712301    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc" podStartSLOduration=6.712276671 podStartE2EDuration="6.712276671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:42:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:43:05.711592998 +0000 UTC m=+1625.627902429" watchObservedRunningTime="2026-10-31 21:43:05.712276671 +0000 UTC m=+1625.628586101"
Oct 31 21:43:06.032078 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:06.032041    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:43:06.038148 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:06.037806    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:43:06.055756 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:06.055725    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:43:06.057011 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:06.056985    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:43:06.345375 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:06.345298    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:43:06.346317 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:06.346293    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:43:06.583165 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:06.583135    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:43:06.584151 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:06.584127    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:43:07.461377 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:07.461334    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:08.110481 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:08.110440    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:08.231197 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:08.231160    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:09.395710 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:09.395678    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:09.623675 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:09.623590    4460 generic.go:356] "Generic (PLEG): container finished" podID="928636a0-a99a-43da-b29f-6ad518f022e1" containerID="b6242167b078c6ed8d290d52e301b250ee50f218aa5dc91fedb6634155510afc" exitCode=0
Oct 31 21:43:09.623826 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:09.623666    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk" event={"ID":"928636a0-a99a-43da-b29f-6ad518f022e1","Type":"ContainerDied","Data":"b6242167b078c6ed8d290d52e301b250ee50f218aa5dc91fedb6634155510afc"}
Oct 31 21:43:09.624162 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:09.624143    4460 scope.go:122] "RemoveContainer" containerID="b6242167b078c6ed8d290d52e301b250ee50f218aa5dc91fedb6634155510afc"
Oct 31 21:43:09.625946 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:09.625903    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:43:09.626015 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:09.625984    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:43:09.626051 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:09.626018    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:43:09.637865 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:09.637833    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:43:10.652745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:10.652698    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk" event={"ID":"928636a0-a99a-43da-b29f-6ad518f022e1","Type":"ContainerStarted","Data":"2e03ed96e4e8538b99401dfa62213aec023284ec897ea0ea8ed19c9a88cbd722"}
Oct 31 21:43:11.053470 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:11.053436    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:43:11.053648 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:11.053606    4460 patch_prober.go:29] interesting pod/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused" start-of-body=
Oct 31 21:43:11.053692 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:11.053650    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk" podUID="928636a0-a99a-43da-b29f-6ad518f022e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused"
Oct 31 21:43:11.390326 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:11.390246    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:11.547667 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:11.547621    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:11.778991 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:11.772135    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:12.061336 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:12.059574    4460 patch_prober.go:29] interesting pod/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused" start-of-body=
Oct 31 21:43:12.061336 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:12.059650    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk" podUID="928636a0-a99a-43da-b29f-6ad518f022e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused"
Oct 31 21:43:12.852723 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:12.852696    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:12.974352 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:12.974318    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:13.071008 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:13.057988    4460 patch_prober.go:29] interesting pod/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused" start-of-body=
Oct 31 21:43:13.130019 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:13.104799    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:13.175966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:13.153415    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk" podUID="928636a0-a99a-43da-b29f-6ad518f022e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused"
Oct 31 21:43:13.908192 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:13.908152    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:14.071966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.068684    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:14.077043 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.076993    4460 patch_prober.go:29] interesting pod/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused" start-of-body=
Oct 31 21:43:14.077330 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.077302    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk" podUID="928636a0-a99a-43da-b29f-6ad518f022e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused"
Oct 31 21:43:14.236296 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.236202    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:14.383653 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.366193    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"]
Oct 31 21:43:14.416417 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.416382    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"]
Oct 31 21:43:14.416735 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.416721    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:43:14.494416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.494346    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-b-9c8b33-openshift-default-dockercfg-kp99f\""
Oct 31 21:43:14.545013 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.520815    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istio-data\") pod \"e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") " pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:43:14.545203 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.545111    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-workload-socket\") pod \"e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") " pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:43:14.545278 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.545237    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istio-token\") pod \"e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") " pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:43:14.545337 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.545305    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-credential-socket\") pod \"e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") " pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:43:14.545516 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.545488    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istio-envoy\") pod \"e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") " pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:43:14.545667 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.545646    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-workload-certs\") pod \"e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") " pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:43:14.545741 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.545678    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istio-podinfo\") pod \"e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") " pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:43:14.545741 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.545710    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istiod-ca-cert\") pod \"e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") " pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:43:14.545838 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.545749    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxff6\" (UniqueName: \"kubernetes.io/projected/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-kube-api-access-xxff6\") pod \"e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") " pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:43:14.662968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.650086    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istio-envoy\") pod \"e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") " pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:43:14.662968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.650348    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-workload-certs\") pod \"e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") " pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:43:14.662968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.650385    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istio-podinfo\") pod \"e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") " pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:43:14.662968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.650433    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istiod-ca-cert\") pod \"e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") " pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:43:14.662968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.650515    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxff6\" (UniqueName: \"kubernetes.io/projected/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-kube-api-access-xxff6\") pod \"e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") " pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:43:14.662968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.650612    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istio-data\") pod \"e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") " pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:43:14.662968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.650744    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-workload-socket\") pod \"e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") " pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:43:14.662968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.650873    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istio-token\") pod \"e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") " pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:43:14.662968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.650934    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-credential-socket\") pod \"e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") " pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:43:14.662968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.652740    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-workload-certs\") pod \"e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") " pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:43:14.662968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.653036    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-workload-socket\") pod \"e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") " pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:43:14.662968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.653655    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istio-data\") pod \"e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") " pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:43:14.666362 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.666003    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istiod-ca-cert\") pod \"e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") " pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:43:14.683603 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.683560    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istio-podinfo\") pod \"e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") " pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:43:14.684950 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.684903    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-credential-socket\") pod \"e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") " pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:43:14.696818 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.696739    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxff6\" (UniqueName: \"kubernetes.io/projected/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-kube-api-access-xxff6\") pod \"e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") " pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:43:14.716979 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.716082    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istio-envoy\") pod \"e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") " pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:43:14.727561 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.727490    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istio-token\") pod \"e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") " pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:43:14.759259 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:14.759172    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:43:15.053037 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:15.053003    4460 patch_prober.go:29] interesting pod/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused" start-of-body=
Oct 31 21:43:15.053414 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:15.053062    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk" podUID="928636a0-a99a-43da-b29f-6ad518f022e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused"
Oct 31 21:43:15.149493 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:15.149451    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"]
Oct 31 21:43:15.151152 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:43:15.151124    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e8f28aa_6c57_4882_92bf_b1c5f9450da5.slice/crio-e2408be4c49a9d91f30e29f4b08305d1213bf26f6b4cf66a4d02cd26c7ec8aa7 WatchSource:0}: Error finding container e2408be4c49a9d91f30e29f4b08305d1213bf26f6b4cf66a4d02cd26c7ec8aa7: Status 404 returned error can't find the container with id e2408be4c49a9d91f30e29f4b08305d1213bf26f6b4cf66a4d02cd26c7ec8aa7
Oct 31 21:43:15.154375 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:15.154341    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:43:15.154572 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:15.154552    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:43:15.154709 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:15.154694    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:43:15.792194 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:15.792155    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj" event={"ID":"4e8f28aa-6c57-4882-92bf-b1c5f9450da5","Type":"ContainerStarted","Data":"a0dd4795966350e151c1d7302266d13aa94c85669f67dcfb49fe143dab683cd8"}
Oct 31 21:43:15.792373 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:15.792215    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj" event={"ID":"4e8f28aa-6c57-4882-92bf-b1c5f9450da5","Type":"ContainerStarted","Data":"e2408be4c49a9d91f30e29f4b08305d1213bf26f6b4cf66a4d02cd26c7ec8aa7"}
Oct 31 21:43:15.877629 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:15.877580    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj" podStartSLOduration=2.877564172 podStartE2EDuration="2.877564172s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:43:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:43:15.875335865 +0000 UTC m=+1635.791645299" watchObservedRunningTime="2026-10-31 21:43:15.877564172 +0000 UTC m=+1635.793873603"
Oct 31 21:43:16.047413 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:16.047331    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:16.053356 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:16.053318    4460 patch_prober.go:29] interesting pod/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused" start-of-body=
Oct 31 21:43:16.053863 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:16.053386    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk" podUID="928636a0-a99a-43da-b29f-6ad518f022e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.121:15021/healthz/ready\": dial tcp 10.133.0.121:15021: connect: connection refused"
Oct 31 21:43:16.099529 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:16.099488    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"]
Oct 31 21:43:16.100041 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:16.100000    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9" podUID="9501cf33-637a-4d00-8e58-3ae157dd6840" containerName="istio-proxy" containerID="cri-o://ff0ac493ac8a0d8677ef67fc5cc9d3069b738c9ef3338f07eb651fb194cc4a8a" gracePeriod=30
Oct 31 21:43:16.762745 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:16.762713    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:43:16.768192 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:16.768164    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:43:16.800080 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:16.800050    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:43:16.800909 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:16.800888    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:43:17.059002 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:17.058810    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:43:17.059639 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:17.059609    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:43:17.060310 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:17.060290    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:43:17.795787 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:17.795754    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:18.060469 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:18.060383    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:18.447901 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:18.447811    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:18.562079 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:18.562038    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:18.817121 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:18.817084    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:20.216009 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:20.215974    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:20.348471 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:20.348427    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:20.406447 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:20.406418    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-9c8b33-576597d459-mq54c"]
Oct 31 21:43:20.410754 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:20.410725    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9c8b33-576597d459-mq54c"
Oct 31 21:43:20.435879 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:20.435846    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-9c8b33-576597d459-mq54c"]
Oct 31 21:43:20.442295 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:20.442267    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-b-9c8b33-dockercfg-948cl\""
Oct 31 21:43:20.442295 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:20.442267    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-b-9c8b33\""
Oct 31 21:43:20.480115 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:20.480027    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:20.516144 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:20.516108    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39674f94-37fe-47b6-b2a0-6449ada9d378-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-9c8b33-576597d459-mq54c\" (UID: \"39674f94-37fe-47b6-b2a0-6449ada9d378\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9c8b33-576597d459-mq54c"
Oct 31 21:43:20.516335 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:20.516279    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t74m4\" (UniqueName: \"kubernetes.io/projected/39674f94-37fe-47b6-b2a0-6449ada9d378-kube-api-access-t74m4\") pod \"payload-pre-processing-e2e-praxis-b-9c8b33-576597d459-mq54c\" (UID: \"39674f94-37fe-47b6-b2a0-6449ada9d378\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9c8b33-576597d459-mq54c"
Oct 31 21:43:20.634254 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:20.634214    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39674f94-37fe-47b6-b2a0-6449ada9d378-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-9c8b33-576597d459-mq54c\" (UID: \"39674f94-37fe-47b6-b2a0-6449ada9d378\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9c8b33-576597d459-mq54c"
Oct 31 21:43:20.634676 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:20.634653    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t74m4\" (UniqueName: \"kubernetes.io/projected/39674f94-37fe-47b6-b2a0-6449ada9d378-kube-api-access-t74m4\") pod \"payload-pre-processing-e2e-praxis-b-9c8b33-576597d459-mq54c\" (UID: \"39674f94-37fe-47b6-b2a0-6449ada9d378\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9c8b33-576597d459-mq54c"
Oct 31 21:43:20.669963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:20.643672    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39674f94-37fe-47b6-b2a0-6449ada9d378-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-9c8b33-576597d459-mq54c\" (UID: \"39674f94-37fe-47b6-b2a0-6449ada9d378\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9c8b33-576597d459-mq54c"
Oct 31 21:43:20.698964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:20.698048    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t74m4\" (UniqueName: \"kubernetes.io/projected/39674f94-37fe-47b6-b2a0-6449ada9d378-kube-api-access-t74m4\") pod \"payload-pre-processing-e2e-praxis-b-9c8b33-576597d459-mq54c\" (UID: \"39674f94-37fe-47b6-b2a0-6449ada9d378\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9c8b33-576597d459-mq54c"
Oct 31 21:43:20.739688 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:20.739590    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9c8b33-576597d459-mq54c"
Oct 31 21:43:21.140598 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.140563    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:21.170202 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.170174    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-9c8b33-576597d459-mq54c"]
Oct 31 21:43:21.191539 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:43:21.191509    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39674f94_37fe_47b6_b2a0_6449ada9d378.slice/crio-5575c966ddde6b3fccf8f1a6293fe8eacf2ffb98c9326b9aa42ad285d3b4e710 WatchSource:0}: Error finding container 5575c966ddde6b3fccf8f1a6293fe8eacf2ffb98c9326b9aa42ad285d3b4e710: Status 404 returned error can't find the container with id 5575c966ddde6b3fccf8f1a6293fe8eacf2ffb98c9326b9aa42ad285d3b4e710
Oct 31 21:43:21.373637 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.373613    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:43:21.459544 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:43:21.459462    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:43:21.464049 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.464029    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-credential-socket\") pod \"9501cf33-637a-4d00-8e58-3ae157dd6840\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") "
Oct 31 21:43:21.464136 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.464097    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-istio-data\") pod \"9501cf33-637a-4d00-8e58-3ae157dd6840\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") "
Oct 31 21:43:21.464206 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.464170    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9501cf33-637a-4d00-8e58-3ae157dd6840-kube-api-access-mjfwb\" (UniqueName: \"kubernetes.io/projected/9501cf33-637a-4d00-8e58-3ae157dd6840-kube-api-access-mjfwb\") pod \"9501cf33-637a-4d00-8e58-3ae157dd6840\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") "
Oct 31 21:43:21.464264 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.464252    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-istio-envoy\") pod \"9501cf33-637a-4d00-8e58-3ae157dd6840\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") "
Oct 31 21:43:21.464344 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.464327    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-workload-certs\") pod \"9501cf33-637a-4d00-8e58-3ae157dd6840\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") "
Oct 31 21:43:21.464403 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.464344    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-credential-socket" pod "9501cf33-637a-4d00-8e58-3ae157dd6840" (UID: "9501cf33-637a-4d00-8e58-3ae157dd6840"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:43:21.464403 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.464360    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-istio-data" pod "9501cf33-637a-4d00-8e58-3ae157dd6840" (UID: "9501cf33-637a-4d00-8e58-3ae157dd6840"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:43:21.464519 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.464417    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-workload-socket\") pod \"9501cf33-637a-4d00-8e58-3ae157dd6840\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") "
Oct 31 21:43:21.464519 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.464452    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9501cf33-637a-4d00-8e58-3ae157dd6840-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9501cf33-637a-4d00-8e58-3ae157dd6840-istio-podinfo\") pod \"9501cf33-637a-4d00-8e58-3ae157dd6840\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") "
Oct 31 21:43:21.464625 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.464516    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9501cf33-637a-4d00-8e58-3ae157dd6840-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9501cf33-637a-4d00-8e58-3ae157dd6840-istiod-ca-cert\") pod \"9501cf33-637a-4d00-8e58-3ae157dd6840\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") "
Oct 31 21:43:21.464625 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.464566    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9501cf33-637a-4d00-8e58-3ae157dd6840-istio-token\" (UniqueName: \"kubernetes.io/projected/9501cf33-637a-4d00-8e58-3ae157dd6840-istio-token\") pod \"9501cf33-637a-4d00-8e58-3ae157dd6840\" (UID: \"9501cf33-637a-4d00-8e58-3ae157dd6840\") "
Oct 31 21:43:21.465138 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.464921    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9501cf33-637a-4d00-8e58-3ae157dd6840-istiod-ca-cert" pod "9501cf33-637a-4d00-8e58-3ae157dd6840" (UID: "9501cf33-637a-4d00-8e58-3ae157dd6840"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:43:21.465138 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.464989    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-workload-socket" pod "9501cf33-637a-4d00-8e58-3ae157dd6840" (UID: "9501cf33-637a-4d00-8e58-3ae157dd6840"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:43:21.465480 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.465461    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:21.465578 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.465485    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:21.465578 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.465498    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:21.465578 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.465510    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9501cf33-637a-4d00-8e58-3ae157dd6840-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:21.465828 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.465803    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-workload-certs" pod "9501cf33-637a-4d00-8e58-3ae157dd6840" (UID: "9501cf33-637a-4d00-8e58-3ae157dd6840"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:43:21.467066 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.467040    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-istio-envoy" pod "9501cf33-637a-4d00-8e58-3ae157dd6840" (UID: "9501cf33-637a-4d00-8e58-3ae157dd6840"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:43:21.467169 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.467077    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9501cf33-637a-4d00-8e58-3ae157dd6840-istio-podinfo" pod "9501cf33-637a-4d00-8e58-3ae157dd6840" (UID: "9501cf33-637a-4d00-8e58-3ae157dd6840"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:43:21.467260 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.467238    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9501cf33-637a-4d00-8e58-3ae157dd6840-kube-api-access-mjfwb" pod "9501cf33-637a-4d00-8e58-3ae157dd6840" (UID: "9501cf33-637a-4d00-8e58-3ae157dd6840"). InnerVolumeSpecName "kube-api-access-mjfwb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:43:21.467633 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.467615    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9501cf33-637a-4d00-8e58-3ae157dd6840-istio-token" pod "9501cf33-637a-4d00-8e58-3ae157dd6840" (UID: "9501cf33-637a-4d00-8e58-3ae157dd6840"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:43:21.567246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.567201    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjfwb\" (UniqueName: \"kubernetes.io/projected/9501cf33-637a-4d00-8e58-3ae157dd6840-kube-api-access-mjfwb\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:21.567246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.567236    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:21.567246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.567249    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9501cf33-637a-4d00-8e58-3ae157dd6840-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:21.567484 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.567261    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9501cf33-637a-4d00-8e58-3ae157dd6840-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:21.567484 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.567272    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9501cf33-637a-4d00-8e58-3ae157dd6840-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:21.848218 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.848177    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9c8b33-576597d459-mq54c" event={"ID":"39674f94-37fe-47b6-b2a0-6449ada9d378","Type":"ContainerStarted","Data":"8e30f3b6fd07eab01a85bcbd64855f362f4f145ce6d1195c2b2dfe9cb0026f8f"}
Oct 31 21:43:21.848386 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.848253    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9c8b33-576597d459-mq54c"
Oct 31 21:43:21.848386 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.848280    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9c8b33-576597d459-mq54c" event={"ID":"39674f94-37fe-47b6-b2a0-6449ada9d378","Type":"ContainerStarted","Data":"5575c966ddde6b3fccf8f1a6293fe8eacf2ffb98c9326b9aa42ad285d3b4e710"}
Oct 31 21:43:21.849880 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.849850    4460 generic.go:356] "Generic (PLEG): container finished" podID="9501cf33-637a-4d00-8e58-3ae157dd6840" containerID="ff0ac493ac8a0d8677ef67fc5cc9d3069b738c9ef3338f07eb651fb194cc4a8a" exitCode=0
Oct 31 21:43:21.850021 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.849893    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"
Oct 31 21:43:21.850021 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.849932    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9" event={"ID":"9501cf33-637a-4d00-8e58-3ae157dd6840","Type":"ContainerDied","Data":"ff0ac493ac8a0d8677ef67fc5cc9d3069b738c9ef3338f07eb651fb194cc4a8a"}
Oct 31 21:43:21.850021 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.849997    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9" event={"ID":"9501cf33-637a-4d00-8e58-3ae157dd6840","Type":"ContainerDied","Data":"66e9431332d3ad3d5a00200b4317b20d291ece95b5cf10536b148dc8e9b3b4d3"}
Oct 31 21:43:21.850140 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.850029    4460 scope.go:122] "RemoveContainer" containerID="ff0ac493ac8a0d8677ef67fc5cc9d3069b738c9ef3338f07eb651fb194cc4a8a"
Oct 31 21:43:21.862879 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.862845    4460 scope.go:122] "RemoveContainer" containerID="e2c939512586f906bab898a18a9dabd22b7a1e5615e0e076315b70cc71573229"
Oct 31 21:43:21.876122 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.876088    4460 scope.go:122] "RemoveContainer" containerID="ff0ac493ac8a0d8677ef67fc5cc9d3069b738c9ef3338f07eb651fb194cc4a8a"
Oct 31 21:43:21.876446 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:43:21.876419    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff0ac493ac8a0d8677ef67fc5cc9d3069b738c9ef3338f07eb651fb194cc4a8a\": container with ID starting with ff0ac493ac8a0d8677ef67fc5cc9d3069b738c9ef3338f07eb651fb194cc4a8a not found: ID does not exist" containerID="ff0ac493ac8a0d8677ef67fc5cc9d3069b738c9ef3338f07eb651fb194cc4a8a"
Oct 31 21:43:21.876536 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.876460    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff0ac493ac8a0d8677ef67fc5cc9d3069b738c9ef3338f07eb651fb194cc4a8a"} err="failed to get container status \"ff0ac493ac8a0d8677ef67fc5cc9d3069b738c9ef3338f07eb651fb194cc4a8a\": rpc error: code = NotFound desc = could not find container \"ff0ac493ac8a0d8677ef67fc5cc9d3069b738c9ef3338f07eb651fb194cc4a8a\": container with ID starting with ff0ac493ac8a0d8677ef67fc5cc9d3069b738c9ef3338f07eb651fb194cc4a8a not found: ID does not exist"
Oct 31 21:43:21.876536 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.876487    4460 scope.go:122] "RemoveContainer" containerID="e2c939512586f906bab898a18a9dabd22b7a1e5615e0e076315b70cc71573229"
Oct 31 21:43:21.876755 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:43:21.876732    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2c939512586f906bab898a18a9dabd22b7a1e5615e0e076315b70cc71573229\": container with ID starting with e2c939512586f906bab898a18a9dabd22b7a1e5615e0e076315b70cc71573229 not found: ID does not exist" containerID="e2c939512586f906bab898a18a9dabd22b7a1e5615e0e076315b70cc71573229"
Oct 31 21:43:21.876821 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.876763    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2c939512586f906bab898a18a9dabd22b7a1e5615e0e076315b70cc71573229"} err="failed to get container status \"e2c939512586f906bab898a18a9dabd22b7a1e5615e0e076315b70cc71573229\": rpc error: code = NotFound desc = could not find container \"e2c939512586f906bab898a18a9dabd22b7a1e5615e0e076315b70cc71573229\": container with ID starting with e2c939512586f906bab898a18a9dabd22b7a1e5615e0e076315b70cc71573229 not found: ID does not exist"
Oct 31 21:43:21.898835 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.898790    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9c8b33-576597d459-mq54c" podStartSLOduration=1.898779035 podStartE2EDuration="1.898779035s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:43:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:43:21.897897294 +0000 UTC m=+1641.814206723" watchObservedRunningTime="2026-10-31 21:43:21.898779035 +0000 UTC m=+1641.815088731"
Oct 31 21:43:21.944659 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.944623    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"]
Oct 31 21:43:21.958153 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:21.958120    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-tqnl9"]
Oct 31 21:43:22.226768 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:22.226691    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9501cf33-637a-4d00-8e58-3ae157dd6840" path="/var/lib/kubelet/pods/9501cf33-637a-4d00-8e58-3ae157dd6840/volumes"
Oct 31 21:43:22.316559 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:22.316526    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:24.145010 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:24.144931    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:25.963263 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:25.963230    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:27.753206 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:27.753164    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:27.911962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:27.908479    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:28.143266 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:28.143232    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:30.045832 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:30.045798    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"]
Oct 31 21:43:30.046258 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:30.046014    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk" podUID="d136f2ed-85d7-4e13-936a-032a993576f0" containerName="istio-proxy" containerID="cri-o://0e8f88b4ca2b74283874bc2e479d26b0c84850bc3bce64596a4941d95956fbbc" gracePeriod=30
Oct 31 21:43:30.067717 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:30.061552    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"]
Oct 31 21:43:30.067717 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:30.061864    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659" podUID="00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a" containerName="istio-proxy" containerID="cri-o://7dea388ed0cfdce2384608dd6431fbe8aeb85ad4504ad1152509d16d97531a8d" gracePeriod=30
Oct 31 21:43:30.253935 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:30.253891    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:31.820457 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:31.820417    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:32.061871 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:32.061834    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:32.351771 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:32.351724    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:32.664131 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:32.664052    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:32.861586 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:32.861558    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9c8b33-576597d459-mq54c"
Oct 31 21:43:34.160514 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:34.160477    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:35.067394 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.067358    4460 patch_prober.go:29] interesting pod/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 21:43:35.067529 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.067422    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659" podUID="00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 21:43:35.071538 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.071511    4460 patch_prober.go:29] interesting pod/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 31 21:43:35.071662 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.071555    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk" podUID="d136f2ed-85d7-4e13-936a-032a993576f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 31 21:43:35.373340 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.373315    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:43:35.390749 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.390724    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:43:35.465013 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.464484    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d136f2ed-85d7-4e13-936a-032a993576f0-kube-api-access-fbklc\" (UniqueName: \"kubernetes.io/projected/d136f2ed-85d7-4e13-936a-032a993576f0-kube-api-access-fbklc\") pod \"d136f2ed-85d7-4e13-936a-032a993576f0\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") "
Oct 31 21:43:35.465013 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.464540    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-workload-socket\") pod \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") "
Oct 31 21:43:35.465013 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.464572    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d136f2ed-85d7-4e13-936a-032a993576f0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d136f2ed-85d7-4e13-936a-032a993576f0-istio-podinfo\") pod \"d136f2ed-85d7-4e13-936a-032a993576f0\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") "
Oct 31 21:43:35.465013 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.464629    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-workload-socket\") pod \"d136f2ed-85d7-4e13-936a-032a993576f0\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") "
Oct 31 21:43:35.465013 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.464669    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d136f2ed-85d7-4e13-936a-032a993576f0-istio-token\" (UniqueName: \"kubernetes.io/projected/d136f2ed-85d7-4e13-936a-032a993576f0-istio-token\") pod \"d136f2ed-85d7-4e13-936a-032a993576f0\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") "
Oct 31 21:43:35.465013 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.464720    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-workload-certs\") pod \"d136f2ed-85d7-4e13-936a-032a993576f0\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") "
Oct 31 21:43:35.465013 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.464774    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-credential-socket\") pod \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") "
Oct 31 21:43:35.465013 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.464837    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d136f2ed-85d7-4e13-936a-032a993576f0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d136f2ed-85d7-4e13-936a-032a993576f0-istiod-ca-cert\") pod \"d136f2ed-85d7-4e13-936a-032a993576f0\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") "
Oct 31 21:43:35.465013 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.464910    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-istio-envoy\") pod \"d136f2ed-85d7-4e13-936a-032a993576f0\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") "
Oct 31 21:43:35.465616 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.465295    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-credential-socket" pod "00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a" (UID: "00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:43:35.465902 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.465718    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-workload-certs" pod "d136f2ed-85d7-4e13-936a-032a993576f0" (UID: "d136f2ed-85d7-4e13-936a-032a993576f0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:43:35.465902 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.465731    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-kube-api-access-gpc8g\" (UniqueName: \"kubernetes.io/projected/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-kube-api-access-gpc8g\") pod \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") "
Oct 31 21:43:35.465902 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.465794    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istio-envoy\") pod \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") "
Oct 31 21:43:35.465902 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.465820    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d136f2ed-85d7-4e13-936a-032a993576f0-istiod-ca-cert" pod "d136f2ed-85d7-4e13-936a-032a993576f0" (UID: "d136f2ed-85d7-4e13-936a-032a993576f0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:43:35.465902 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.465843    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-istio-data\") pod \"d136f2ed-85d7-4e13-936a-032a993576f0\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") "
Oct 31 21:43:35.465902 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.465899    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-workload-certs\") pod \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") "
Oct 31 21:43:35.466310 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.466014    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-credential-socket\") pod \"d136f2ed-85d7-4e13-936a-032a993576f0\" (UID: \"d136f2ed-85d7-4e13-936a-032a993576f0\") "
Oct 31 21:43:35.466310 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.466109    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istio-podinfo\") pod \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") "
Oct 31 21:43:35.466310 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.466180    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istiod-ca-cert\") pod \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") "
Oct 31 21:43:35.466310 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.466231    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istio-token\" (UniqueName: \"kubernetes.io/projected/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istio-token\") pod \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") "
Oct 31 21:43:35.466310 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.466265    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istio-data\") pod \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\" (UID: \"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a\") "
Oct 31 21:43:35.466972 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.466932    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-workload-socket" pod "d136f2ed-85d7-4e13-936a-032a993576f0" (UID: "d136f2ed-85d7-4e13-936a-032a993576f0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:43:35.467152 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.467127    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-workload-certs" pod "00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a" (UID: "00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:43:35.467361 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.467327    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-workload-socket" pod "00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a" (UID: "00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:43:35.467477 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.467456    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istiod-ca-cert" pod "00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a" (UID: "00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:43:35.467794 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.467745    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-credential-socket" pod "d136f2ed-85d7-4e13-936a-032a993576f0" (UID: "d136f2ed-85d7-4e13-936a-032a993576f0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:43:35.468123 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.468099    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:35.468214 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.468129    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:35.468214 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.468145    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:35.468214 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.468159    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:35.468214 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.468173    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:35.468214 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.468186    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d136f2ed-85d7-4e13-936a-032a993576f0-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:35.468214 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.468199    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:35.468214 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.468211    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:35.468624 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.468273    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d136f2ed-85d7-4e13-936a-032a993576f0-istio-token" pod "d136f2ed-85d7-4e13-936a-032a993576f0" (UID: "d136f2ed-85d7-4e13-936a-032a993576f0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:43:35.468624 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.468583    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istio-data" pod "00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a" (UID: "00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:43:35.468734 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.468632    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-istio-data" pod "d136f2ed-85d7-4e13-936a-032a993576f0" (UID: "d136f2ed-85d7-4e13-936a-032a993576f0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:43:35.469314 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.469277    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d136f2ed-85d7-4e13-936a-032a993576f0-kube-api-access-fbklc" pod "d136f2ed-85d7-4e13-936a-032a993576f0" (UID: "d136f2ed-85d7-4e13-936a-032a993576f0"). InnerVolumeSpecName "kube-api-access-fbklc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:43:35.469617 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.469579    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istio-token" pod "00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a" (UID: "00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:43:35.469852 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.469828    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istio-envoy" pod "00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a" (UID: "00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:43:35.470132 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.470109    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-kube-api-access-gpc8g" pod "00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a" (UID: "00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a"). InnerVolumeSpecName "kube-api-access-gpc8g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:43:35.471252 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.471229    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istio-podinfo" pod "00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a" (UID: "00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:43:35.472121 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.472094    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-istio-envoy" pod "d136f2ed-85d7-4e13-936a-032a993576f0" (UID: "d136f2ed-85d7-4e13-936a-032a993576f0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:43:35.474581 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.474542    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d136f2ed-85d7-4e13-936a-032a993576f0-istio-podinfo" pod "d136f2ed-85d7-4e13-936a-032a993576f0" (UID: "d136f2ed-85d7-4e13-936a-032a993576f0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:43:35.569271 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.569235    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:35.569271 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.569264    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gpc8g\" (UniqueName: \"kubernetes.io/projected/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-kube-api-access-gpc8g\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:35.569271 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.569274    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:35.569271 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.569283    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d136f2ed-85d7-4e13-936a-032a993576f0-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:35.569535 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.569292    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:35.569535 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.569301    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:35.569535 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.569309    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:35.569535 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.569317    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbklc\" (UniqueName: \"kubernetes.io/projected/d136f2ed-85d7-4e13-936a-032a993576f0-kube-api-access-fbklc\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:35.569535 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.569325    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d136f2ed-85d7-4e13-936a-032a993576f0-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:35.569535 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.569332    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d136f2ed-85d7-4e13-936a-032a993576f0-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:35.986804 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.986767    4460 generic.go:356] "Generic (PLEG): container finished" podID="00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a" containerID="7dea388ed0cfdce2384608dd6431fbe8aeb85ad4504ad1152509d16d97531a8d" exitCode=0
Oct 31 21:43:35.987031 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.986820    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"
Oct 31 21:43:35.987031 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.986852    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659" event={"ID":"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a","Type":"ContainerDied","Data":"7dea388ed0cfdce2384608dd6431fbe8aeb85ad4504ad1152509d16d97531a8d"}
Oct 31 21:43:35.987031 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.986908    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659" event={"ID":"00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a","Type":"ContainerDied","Data":"be22e7774a11b8d18b70095a0a169a288043d0892a07eb7f6f784377b1c8e160"}
Oct 31 21:43:35.987031 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.986968    4460 scope.go:122] "RemoveContainer" containerID="7dea388ed0cfdce2384608dd6431fbe8aeb85ad4504ad1152509d16d97531a8d"
Oct 31 21:43:35.988449 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.988420    4460 generic.go:356] "Generic (PLEG): container finished" podID="d136f2ed-85d7-4e13-936a-032a993576f0" containerID="0e8f88b4ca2b74283874bc2e479d26b0c84850bc3bce64596a4941d95956fbbc" exitCode=0
Oct 31 21:43:35.988554 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.988455    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk" event={"ID":"d136f2ed-85d7-4e13-936a-032a993576f0","Type":"ContainerDied","Data":"0e8f88b4ca2b74283874bc2e479d26b0c84850bc3bce64596a4941d95956fbbc"}
Oct 31 21:43:35.988554 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.988482    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"
Oct 31 21:43:35.988554 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:35.988498    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk" event={"ID":"d136f2ed-85d7-4e13-936a-032a993576f0","Type":"ContainerDied","Data":"0e4ef95bbcffd650bc967ed73f2afe2290206633417988c0df9631ffd4d05913"}
Oct 31 21:43:36.000051 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.000033    4460 scope.go:122] "RemoveContainer" containerID="7dea388ed0cfdce2384608dd6431fbe8aeb85ad4504ad1152509d16d97531a8d"
Oct 31 21:43:36.000302 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:43:36.000280    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7dea388ed0cfdce2384608dd6431fbe8aeb85ad4504ad1152509d16d97531a8d\": container with ID starting with 7dea388ed0cfdce2384608dd6431fbe8aeb85ad4504ad1152509d16d97531a8d not found: ID does not exist" containerID="7dea388ed0cfdce2384608dd6431fbe8aeb85ad4504ad1152509d16d97531a8d"
Oct 31 21:43:36.000366 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.000311    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dea388ed0cfdce2384608dd6431fbe8aeb85ad4504ad1152509d16d97531a8d"} err="failed to get container status \"7dea388ed0cfdce2384608dd6431fbe8aeb85ad4504ad1152509d16d97531a8d\": rpc error: code = NotFound desc = could not find container \"7dea388ed0cfdce2384608dd6431fbe8aeb85ad4504ad1152509d16d97531a8d\": container with ID starting with 7dea388ed0cfdce2384608dd6431fbe8aeb85ad4504ad1152509d16d97531a8d not found: ID does not exist"
Oct 31 21:43:36.000366 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.000327    4460 scope.go:122] "RemoveContainer" containerID="0e8f88b4ca2b74283874bc2e479d26b0c84850bc3bce64596a4941d95956fbbc"
Oct 31 21:43:36.010052 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.010033    4460 scope.go:122] "RemoveContainer" containerID="0e8f88b4ca2b74283874bc2e479d26b0c84850bc3bce64596a4941d95956fbbc"
Oct 31 21:43:36.010302 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:43:36.010286    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e8f88b4ca2b74283874bc2e479d26b0c84850bc3bce64596a4941d95956fbbc\": container with ID starting with 0e8f88b4ca2b74283874bc2e479d26b0c84850bc3bce64596a4941d95956fbbc not found: ID does not exist" containerID="0e8f88b4ca2b74283874bc2e479d26b0c84850bc3bce64596a4941d95956fbbc"
Oct 31 21:43:36.010350 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.010309    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e8f88b4ca2b74283874bc2e479d26b0c84850bc3bce64596a4941d95956fbbc"} err="failed to get container status \"0e8f88b4ca2b74283874bc2e479d26b0c84850bc3bce64596a4941d95956fbbc\": rpc error: code = NotFound desc = could not find container \"0e8f88b4ca2b74283874bc2e479d26b0c84850bc3bce64596a4941d95956fbbc\": container with ID starting with 0e8f88b4ca2b74283874bc2e479d26b0c84850bc3bce64596a4941d95956fbbc not found: ID does not exist"
Oct 31 21:43:36.070358 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.070330    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"]
Oct 31 21:43:36.083456 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.083430    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-fr8nk"]
Oct 31 21:43:36.121317 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.121291    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"]
Oct 31 21:43:36.124800 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.124777    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-tt659"]
Oct 31 21:43:36.226363 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.226332    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a" path="/var/lib/kubelet/pods/00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a/volumes"
Oct 31 21:43:36.227092 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.227070    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d136f2ed-85d7-4e13-936a-032a993576f0" path="/var/lib/kubelet/pods/d136f2ed-85d7-4e13-936a-032a993576f0/volumes"
Oct 31 21:43:36.350358 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.350330    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm"]
Oct 31 21:43:36.351042 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.351026    4460 cpu_manager.go:414] "containerMap: removing container" podUID="9501cf33-637a-4d00-8e58-3ae157dd6840" containerName="istio-proxy"
Oct 31 21:43:36.351114 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.351042    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9501cf33-637a-4d00-8e58-3ae157dd6840" containerName="istio-proxy"
Oct 31 21:43:36.351114 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.351058    4460 cpu_manager.go:414] "containerMap: removing container" podUID="9501cf33-637a-4d00-8e58-3ae157dd6840" containerName="istio-proxy"
Oct 31 21:43:36.351114 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.351064    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9501cf33-637a-4d00-8e58-3ae157dd6840" containerName="istio-proxy"
Oct 31 21:43:36.351114 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.351086    4460 cpu_manager.go:414] "containerMap: removing container" podUID="00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a" containerName="istio-proxy"
Oct 31 21:43:36.351114 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.351091    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a" containerName="istio-proxy"
Oct 31 21:43:36.351114 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.351106    4460 cpu_manager.go:414] "containerMap: removing container" podUID="d136f2ed-85d7-4e13-936a-032a993576f0" containerName="istio-proxy"
Oct 31 21:43:36.351114 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.351111    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d136f2ed-85d7-4e13-936a-032a993576f0" containerName="istio-proxy"
Oct 31 21:43:36.351328 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.351214    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="9501cf33-637a-4d00-8e58-3ae157dd6840" containerName="istio-proxy"
Oct 31 21:43:36.351328 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.351225    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="9501cf33-637a-4d00-8e58-3ae157dd6840" containerName="istio-proxy"
Oct 31 21:43:36.351328 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.351235    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="00f0f1bb-f3ba-4eb0-9fba-147fb2e3d50a" containerName="istio-proxy"
Oct 31 21:43:36.351328 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.351243    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="d136f2ed-85d7-4e13-936a-032a993576f0" containerName="istio-proxy"
Oct 31 21:43:36.356399 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.356380    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm"
Oct 31 21:43:36.375987 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.373528    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-b-f4569e-serving-cert\""
Oct 31 21:43:36.377154 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.377129    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm"]
Oct 31 21:43:36.486735 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.486705    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db04d59d-9b82-492d-b5dd-58d51b35d555-metrics-certs\") pod \"maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm\" (UID: \"db04d59d-9b82-492d-b5dd-58d51b35d555\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm"
Oct 31 21:43:36.486933 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.486759    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xghgr\" (UniqueName: \"kubernetes.io/projected/db04d59d-9b82-492d-b5dd-58d51b35d555-kube-api-access-xghgr\") pod \"maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm\" (UID: \"db04d59d-9b82-492d-b5dd-58d51b35d555\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm"
Oct 31 21:43:36.486933 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.486858    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db04d59d-9b82-492d-b5dd-58d51b35d555-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm\" (UID: \"db04d59d-9b82-492d-b5dd-58d51b35d555\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm"
Oct 31 21:43:36.486933 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.486925    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db04d59d-9b82-492d-b5dd-58d51b35d555-maas-api-tls\") pod \"maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm\" (UID: \"db04d59d-9b82-492d-b5dd-58d51b35d555\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm"
Oct 31 21:43:36.499874 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.499848    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:36.589706 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.589081    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db04d59d-9b82-492d-b5dd-58d51b35d555-maas-api-tls\") pod \"maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm\" (UID: \"db04d59d-9b82-492d-b5dd-58d51b35d555\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm"
Oct 31 21:43:36.589706 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.589348    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db04d59d-9b82-492d-b5dd-58d51b35d555-metrics-certs\") pod \"maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm\" (UID: \"db04d59d-9b82-492d-b5dd-58d51b35d555\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm"
Oct 31 21:43:36.589706 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.589416    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xghgr\" (UniqueName: \"kubernetes.io/projected/db04d59d-9b82-492d-b5dd-58d51b35d555-kube-api-access-xghgr\") pod \"maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm\" (UID: \"db04d59d-9b82-492d-b5dd-58d51b35d555\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm"
Oct 31 21:43:36.589706 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.589482    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db04d59d-9b82-492d-b5dd-58d51b35d555-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm\" (UID: \"db04d59d-9b82-492d-b5dd-58d51b35d555\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm"
Oct 31 21:43:36.598095 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.594913    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db04d59d-9b82-492d-b5dd-58d51b35d555-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm\" (UID: \"db04d59d-9b82-492d-b5dd-58d51b35d555\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm"
Oct 31 21:43:36.611014 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.605015    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db04d59d-9b82-492d-b5dd-58d51b35d555-metrics-certs\") pod \"maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm\" (UID: \"db04d59d-9b82-492d-b5dd-58d51b35d555\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm"
Oct 31 21:43:36.611014 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.605501    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db04d59d-9b82-492d-b5dd-58d51b35d555-maas-api-tls\") pod \"maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm\" (UID: \"db04d59d-9b82-492d-b5dd-58d51b35d555\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm"
Oct 31 21:43:36.652984 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.652030    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:36.652984 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.652154    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xghgr\" (UniqueName: \"kubernetes.io/projected/db04d59d-9b82-492d-b5dd-58d51b35d555-kube-api-access-xghgr\") pod \"maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm\" (UID: \"db04d59d-9b82-492d-b5dd-58d51b35d555\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm"
Oct 31 21:43:36.679457 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.679418    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm"
Oct 31 21:43:36.774922 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.774888    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:36.995357 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:36.995329    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm"]
Oct 31 21:43:36.996063 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:43:36.996040    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb04d59d_9b82_492d_b5dd_58d51b35d555.slice/crio-4c775dea4f5418173d08dbc2aa88f733341b6b6cb3b8ca6c204eb337678f4e44 WatchSource:0}: Error finding container 4c775dea4f5418173d08dbc2aa88f733341b6b6cb3b8ca6c204eb337678f4e44: Status 404 returned error can't find the container with id 4c775dea4f5418173d08dbc2aa88f733341b6b6cb3b8ca6c204eb337678f4e44
Oct 31 21:43:37.396142 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:37.396115    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:38.006457 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:38.006421    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm" event={"ID":"db04d59d-9b82-492d-b5dd-58d51b35d555","Type":"ContainerStarted","Data":"39715ef36f593cc626474aeb7efa425d7386fb591d39348090cfcfee034ed05a"}
Oct 31 21:43:38.006674 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:38.006503    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm" event={"ID":"db04d59d-9b82-492d-b5dd-58d51b35d555","Type":"ContainerStarted","Data":"4c775dea4f5418173d08dbc2aa88f733341b6b6cb3b8ca6c204eb337678f4e44"}
Oct 31 21:43:38.006674 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:38.006630    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm"
Oct 31 21:43:38.054022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:38.053935    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm" podStartSLOduration=1.7161505830000001 podStartE2EDuration="2.053920466s" totalImagesPullingTime="337.769883ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:43:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:43:38.051758833 +0000 UTC m=+1657.968068261" watchObservedRunningTime="2026-10-31 21:43:38.053920466 +0000 UTC m=+1657.970229894"
Oct 31 21:43:38.238980 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:38.236206    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"]
Oct 31 21:43:38.238980 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:38.236408    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78" podUID="7e7288b0-58f2-41e8-a26e-eddac51598e0" containerName="istio-proxy" containerID="cri-o://849a42cd2a5e199c7923c92d4ee4a075cf62e57926d39bee73a112da58cab459" gracePeriod=30
Oct 31 21:43:38.519093 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:38.519058    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:38.655808 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:38.655778    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:39.053688 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:39.052807    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:41.056068 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:41.056033    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:42.955066 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:42.954977    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:43.505435 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:43.505412    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:43:43.542696 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:43.542662    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7e7288b0-58f2-41e8-a26e-eddac51598e0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e7288b0-58f2-41e8-a26e-eddac51598e0-istiod-ca-cert\") pod \"7e7288b0-58f2-41e8-a26e-eddac51598e0\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") "
Oct 31 21:43:43.542852 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:43.542718    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-istio-data\") pod \"7e7288b0-58f2-41e8-a26e-eddac51598e0\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") "
Oct 31 21:43:43.542852 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:43.542748    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-credential-socket\") pod \"7e7288b0-58f2-41e8-a26e-eddac51598e0\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") "
Oct 31 21:43:43.542852 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:43.542813    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-workload-certs\") pod \"7e7288b0-58f2-41e8-a26e-eddac51598e0\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") "
Oct 31 21:43:43.543125 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:43.542909    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-istio-envoy\") pod \"7e7288b0-58f2-41e8-a26e-eddac51598e0\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") "
Oct 31 21:43:43.543125 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:43.543065    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7e7288b0-58f2-41e8-a26e-eddac51598e0-istio-token\" (UniqueName: \"kubernetes.io/projected/7e7288b0-58f2-41e8-a26e-eddac51598e0-istio-token\") pod \"7e7288b0-58f2-41e8-a26e-eddac51598e0\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") "
Oct 31 21:43:43.543236 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:43.543144    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-workload-socket\") pod \"7e7288b0-58f2-41e8-a26e-eddac51598e0\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") "
Oct 31 21:43:43.543236 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:43.543220    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7e7288b0-58f2-41e8-a26e-eddac51598e0-kube-api-access-7xvhm\" (UniqueName: \"kubernetes.io/projected/7e7288b0-58f2-41e8-a26e-eddac51598e0-kube-api-access-7xvhm\") pod \"7e7288b0-58f2-41e8-a26e-eddac51598e0\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") "
Oct 31 21:43:43.543349 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:43.543235    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e7288b0-58f2-41e8-a26e-eddac51598e0-istiod-ca-cert" pod "7e7288b0-58f2-41e8-a26e-eddac51598e0" (UID: "7e7288b0-58f2-41e8-a26e-eddac51598e0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:43:43.543349 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:43.543274    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7e7288b0-58f2-41e8-a26e-eddac51598e0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e7288b0-58f2-41e8-a26e-eddac51598e0-istio-podinfo\") pod \"7e7288b0-58f2-41e8-a26e-eddac51598e0\" (UID: \"7e7288b0-58f2-41e8-a26e-eddac51598e0\") "
Oct 31 21:43:43.544342 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:43.543633    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-workload-socket" pod "7e7288b0-58f2-41e8-a26e-eddac51598e0" (UID: "7e7288b0-58f2-41e8-a26e-eddac51598e0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:43:43.544342 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:43.543699    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-credential-socket" pod "7e7288b0-58f2-41e8-a26e-eddac51598e0" (UID: "7e7288b0-58f2-41e8-a26e-eddac51598e0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:43:43.544342 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:43.543762    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-istio-data" pod "7e7288b0-58f2-41e8-a26e-eddac51598e0" (UID: "7e7288b0-58f2-41e8-a26e-eddac51598e0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:43:43.544342 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:43.543835    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-workload-certs" pod "7e7288b0-58f2-41e8-a26e-eddac51598e0" (UID: "7e7288b0-58f2-41e8-a26e-eddac51598e0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:43:43.544342 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:43.544209    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:43.544342 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:43.544233    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e7288b0-58f2-41e8-a26e-eddac51598e0-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:43.544342 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:43.544246    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:43.544342 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:43.544258    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:43.544342 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:43.544272    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:43.545989 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:43.545926    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7e7288b0-58f2-41e8-a26e-eddac51598e0-istio-podinfo" pod "7e7288b0-58f2-41e8-a26e-eddac51598e0" (UID: "7e7288b0-58f2-41e8-a26e-eddac51598e0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:43:43.546258 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:43.546233    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e7288b0-58f2-41e8-a26e-eddac51598e0-kube-api-access-7xvhm" pod "7e7288b0-58f2-41e8-a26e-eddac51598e0" (UID: "7e7288b0-58f2-41e8-a26e-eddac51598e0"). InnerVolumeSpecName "kube-api-access-7xvhm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:43:43.546372 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:43.546266    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e7288b0-58f2-41e8-a26e-eddac51598e0-istio-token" pod "7e7288b0-58f2-41e8-a26e-eddac51598e0" (UID: "7e7288b0-58f2-41e8-a26e-eddac51598e0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:43:43.546502 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:43.546482    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-istio-envoy" pod "7e7288b0-58f2-41e8-a26e-eddac51598e0" (UID: "7e7288b0-58f2-41e8-a26e-eddac51598e0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:43:43.645443 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:43.645411    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xvhm\" (UniqueName: \"kubernetes.io/projected/7e7288b0-58f2-41e8-a26e-eddac51598e0-kube-api-access-7xvhm\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:43.645443 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:43.645438    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e7288b0-58f2-41e8-a26e-eddac51598e0-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:43.645443 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:43.645447    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e7288b0-58f2-41e8-a26e-eddac51598e0-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:43.645725 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:43.645456    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e7288b0-58f2-41e8-a26e-eddac51598e0-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:43:44.045569 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:44.045541    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm"
Oct 31 21:43:44.071519 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:44.071491    4460 generic.go:356] "Generic (PLEG): container finished" podID="7e7288b0-58f2-41e8-a26e-eddac51598e0" containerID="849a42cd2a5e199c7923c92d4ee4a075cf62e57926d39bee73a112da58cab459" exitCode=0
Oct 31 21:43:44.071679 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:44.071534    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"
Oct 31 21:43:44.071679 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:44.071570    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78" event={"ID":"7e7288b0-58f2-41e8-a26e-eddac51598e0","Type":"ContainerDied","Data":"849a42cd2a5e199c7923c92d4ee4a075cf62e57926d39bee73a112da58cab459"}
Oct 31 21:43:44.071679 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:44.071633    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78" event={"ID":"7e7288b0-58f2-41e8-a26e-eddac51598e0","Type":"ContainerDied","Data":"48dfe40325087e6d6d2ca92599677fdee0559904bedfd56f8e5a6acc0544b1f0"}
Oct 31 21:43:44.071679 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:44.071663    4460 scope.go:122] "RemoveContainer" containerID="849a42cd2a5e199c7923c92d4ee4a075cf62e57926d39bee73a112da58cab459"
Oct 31 21:43:44.082430 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:44.082409    4460 scope.go:122] "RemoveContainer" containerID="849a42cd2a5e199c7923c92d4ee4a075cf62e57926d39bee73a112da58cab459"
Oct 31 21:43:44.082697 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:43:44.082678    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"849a42cd2a5e199c7923c92d4ee4a075cf62e57926d39bee73a112da58cab459\": container with ID starting with 849a42cd2a5e199c7923c92d4ee4a075cf62e57926d39bee73a112da58cab459 not found: ID does not exist" containerID="849a42cd2a5e199c7923c92d4ee4a075cf62e57926d39bee73a112da58cab459"
Oct 31 21:43:44.082754 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:44.082706    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"849a42cd2a5e199c7923c92d4ee4a075cf62e57926d39bee73a112da58cab459"} err="failed to get container status \"849a42cd2a5e199c7923c92d4ee4a075cf62e57926d39bee73a112da58cab459\": rpc error: code = NotFound desc = could not find container \"849a42cd2a5e199c7923c92d4ee4a075cf62e57926d39bee73a112da58cab459\": container with ID starting with 849a42cd2a5e199c7923c92d4ee4a075cf62e57926d39bee73a112da58cab459 not found: ID does not exist"
Oct 31 21:43:44.137528 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:44.137486    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"]
Oct 31 21:43:44.149984 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:44.149949    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-mnz78"]
Oct 31 21:43:44.226430 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:44.226393    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7e7288b0-58f2-41e8-a26e-eddac51598e0" path="/var/lib/kubelet/pods/7e7288b0-58f2-41e8-a26e-eddac51598e0/volumes"
Oct 31 21:43:45.644740 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:45.644699    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:46.880134 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:46.880083    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:47.533429 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:47.533391    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:49.314311 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:49.314277    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:49.430571 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:49.430536    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:49.553540 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:49.550164    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:50.756534 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:50.756502    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:43:51.240456 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:51.240422    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:51.453887 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:43:51.453848    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:43:52.942399 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:52.942366    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:53.501039 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:53.501010    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:53.614143 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:53.614111    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:54.460123 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:54.459901    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:54.749133 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:54.749043    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"]
Oct 31 21:43:54.749682 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:54.749621    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh" podUID="692f0926-1719-44a1-94a9-12604a64cf40" containerName="istio-proxy" containerID="cri-o://e22db340d79a599478fbd492503655a4a93f4c28ea4b2e7f1ab81155af3c21e4" gracePeriod=30
Oct 31 21:43:55.053660 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:55.053570    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:55.171593 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:55.171561    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:55.315062 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:55.313537    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:55.679310 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:55.679214    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:57.010199 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:57.010160    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:58.526891 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:58.526855    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:58.727059 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:58.727018    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:58.837808 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:58.837721    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:59.231644 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:59.231543    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:59.339820 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:59.339780    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:59.449581 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:59.449537    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:43:59.935601 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:43:59.935571    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:00.089753 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.089688    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:44:00.164286 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.164251    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-istio-envoy\") pod \"692f0926-1719-44a1-94a9-12604a64cf40\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") "
Oct 31 21:44:00.164505 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.164394    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/692f0926-1719-44a1-94a9-12604a64cf40-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/692f0926-1719-44a1-94a9-12604a64cf40-istio-podinfo\") pod \"692f0926-1719-44a1-94a9-12604a64cf40\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") "
Oct 31 21:44:00.164505 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.164424    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-workload-certs\") pod \"692f0926-1719-44a1-94a9-12604a64cf40\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") "
Oct 31 21:44:00.164505 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.164470    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-workload-socket\") pod \"692f0926-1719-44a1-94a9-12604a64cf40\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") "
Oct 31 21:44:00.164954 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.164587    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/692f0926-1719-44a1-94a9-12604a64cf40-istio-token\" (UniqueName: \"kubernetes.io/projected/692f0926-1719-44a1-94a9-12604a64cf40-istio-token\") pod \"692f0926-1719-44a1-94a9-12604a64cf40\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") "
Oct 31 21:44:00.164954 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.164628    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/692f0926-1719-44a1-94a9-12604a64cf40-kube-api-access-q4hrh\" (UniqueName: \"kubernetes.io/projected/692f0926-1719-44a1-94a9-12604a64cf40-kube-api-access-q4hrh\") pod \"692f0926-1719-44a1-94a9-12604a64cf40\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") "
Oct 31 21:44:00.164954 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.164716    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-credential-socket\") pod \"692f0926-1719-44a1-94a9-12604a64cf40\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") "
Oct 31 21:44:00.164954 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.164797    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-istio-data\") pod \"692f0926-1719-44a1-94a9-12604a64cf40\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") "
Oct 31 21:44:00.164954 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.164838    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/692f0926-1719-44a1-94a9-12604a64cf40-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/692f0926-1719-44a1-94a9-12604a64cf40-istiod-ca-cert\") pod \"692f0926-1719-44a1-94a9-12604a64cf40\" (UID: \"692f0926-1719-44a1-94a9-12604a64cf40\") "
Oct 31 21:44:00.165601 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.165448    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-workload-socket" pod "692f0926-1719-44a1-94a9-12604a64cf40" (UID: "692f0926-1719-44a1-94a9-12604a64cf40"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:44:00.165856 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.165833    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:44:00.166166 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.166130    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-workload-certs" pod "692f0926-1719-44a1-94a9-12604a64cf40" (UID: "692f0926-1719-44a1-94a9-12604a64cf40"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:44:00.166783 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.166536    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-credential-socket" pod "692f0926-1719-44a1-94a9-12604a64cf40" (UID: "692f0926-1719-44a1-94a9-12604a64cf40"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:44:00.166783 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.166588    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/692f0926-1719-44a1-94a9-12604a64cf40-istiod-ca-cert" pod "692f0926-1719-44a1-94a9-12604a64cf40" (UID: "692f0926-1719-44a1-94a9-12604a64cf40"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:44:00.167401 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.167369    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-istio-data" pod "692f0926-1719-44a1-94a9-12604a64cf40" (UID: "692f0926-1719-44a1-94a9-12604a64cf40"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:44:00.170178 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.170149    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/692f0926-1719-44a1-94a9-12604a64cf40-kube-api-access-q4hrh" pod "692f0926-1719-44a1-94a9-12604a64cf40" (UID: "692f0926-1719-44a1-94a9-12604a64cf40"). InnerVolumeSpecName "kube-api-access-q4hrh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:44:00.170370 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.170350    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-istio-envoy" pod "692f0926-1719-44a1-94a9-12604a64cf40" (UID: "692f0926-1719-44a1-94a9-12604a64cf40"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:44:00.172831 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.172804    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/692f0926-1719-44a1-94a9-12604a64cf40-istio-token" pod "692f0926-1719-44a1-94a9-12604a64cf40" (UID: "692f0926-1719-44a1-94a9-12604a64cf40"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:44:00.173026 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.172930    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/692f0926-1719-44a1-94a9-12604a64cf40-istio-podinfo" pod "692f0926-1719-44a1-94a9-12604a64cf40" (UID: "692f0926-1719-44a1-94a9-12604a64cf40"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:44:00.247336 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.247300    4460 generic.go:356] "Generic (PLEG): container finished" podID="692f0926-1719-44a1-94a9-12604a64cf40" containerID="e22db340d79a599478fbd492503655a4a93f4c28ea4b2e7f1ab81155af3c21e4" exitCode=0
Oct 31 21:44:00.247492 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.247366    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh" event={"ID":"692f0926-1719-44a1-94a9-12604a64cf40","Type":"ContainerDied","Data":"e22db340d79a599478fbd492503655a4a93f4c28ea4b2e7f1ab81155af3c21e4"}
Oct 31 21:44:00.247492 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.247413    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"
Oct 31 21:44:00.247492 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.247432    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh" event={"ID":"692f0926-1719-44a1-94a9-12604a64cf40","Type":"ContainerDied","Data":"74448adf34340444036ac98b1ea48a59f0f765984347060429584887d4d6dec1"}
Oct 31 21:44:00.247492 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.247465    4460 scope.go:122] "RemoveContainer" containerID="e22db340d79a599478fbd492503655a4a93f4c28ea4b2e7f1ab81155af3c21e4"
Oct 31 21:44:00.257519 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.257498    4460 scope.go:122] "RemoveContainer" containerID="e22db340d79a599478fbd492503655a4a93f4c28ea4b2e7f1ab81155af3c21e4"
Oct 31 21:44:00.257803 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:44:00.257783    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e22db340d79a599478fbd492503655a4a93f4c28ea4b2e7f1ab81155af3c21e4\": container with ID starting with e22db340d79a599478fbd492503655a4a93f4c28ea4b2e7f1ab81155af3c21e4 not found: ID does not exist" containerID="e22db340d79a599478fbd492503655a4a93f4c28ea4b2e7f1ab81155af3c21e4"
Oct 31 21:44:00.257891 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.257809    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e22db340d79a599478fbd492503655a4a93f4c28ea4b2e7f1ab81155af3c21e4"} err="failed to get container status \"e22db340d79a599478fbd492503655a4a93f4c28ea4b2e7f1ab81155af3c21e4\": rpc error: code = NotFound desc = could not find container \"e22db340d79a599478fbd492503655a4a93f4c28ea4b2e7f1ab81155af3c21e4\": container with ID starting with e22db340d79a599478fbd492503655a4a93f4c28ea4b2e7f1ab81155af3c21e4 not found: ID does not exist"
Oct 31 21:44:00.280583 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.280275    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:44:00.280583 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.280310    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/692f0926-1719-44a1-94a9-12604a64cf40-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:44:00.280583 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.280323    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:44:00.280583 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.280337    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/692f0926-1719-44a1-94a9-12604a64cf40-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:44:00.280583 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.280352    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4hrh\" (UniqueName: \"kubernetes.io/projected/692f0926-1719-44a1-94a9-12604a64cf40-kube-api-access-q4hrh\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:44:00.280583 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.280367    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:44:00.280583 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.280379    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/692f0926-1719-44a1-94a9-12604a64cf40-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:44:00.280583 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.280392    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/692f0926-1719-44a1-94a9-12604a64cf40-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:44:00.313543 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.313511    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"]
Oct 31 21:44:00.317808 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:00.317756    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-6xsgh"]
Oct 31 21:44:01.259900 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:01.259859    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"]
Oct 31 21:44:01.262457 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:01.260099    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85" podUID="475195a2-42a8-493d-ab68-585d187a977d" containerName="istio-proxy" containerID="cri-o://6ce9e89101e01a48e317553d74bcc248326d0601a67cc7885defba4e1fef4d36" gracePeriod=30
Oct 31 21:44:01.470973 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:01.456487    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:02.225374 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:02.225340    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="692f0926-1719-44a1-94a9-12604a64cf40" path="/var/lib/kubelet/pods/692f0926-1719-44a1-94a9-12604a64cf40/volumes"
Oct 31 21:44:02.263624 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:02.263593    4460 generic.go:356] "Generic (PLEG): container finished" podID="19fcad1c-5c6e-4274-936d-bf93ca76c7d4" containerID="2107324f230fcb14ac11586aff5b1c9a7693440ded3798f4910f5296f1c59b06" exitCode=0
Oct 31 21:44:02.264039 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:02.263664    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc" event={"ID":"19fcad1c-5c6e-4274-936d-bf93ca76c7d4","Type":"ContainerDied","Data":"2107324f230fcb14ac11586aff5b1c9a7693440ded3798f4910f5296f1c59b06"}
Oct 31 21:44:02.264142 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:02.264123    4460 scope.go:122] "RemoveContainer" containerID="2107324f230fcb14ac11586aff5b1c9a7693440ded3798f4910f5296f1c59b06"
Oct 31 21:44:02.265743 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:02.265720    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:44:02.265805 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:02.265784    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:44:02.265841 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:02.265815    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:44:02.627688 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:02.627654    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:02.740082 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:02.740046    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:02.967406 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:02.967313    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"]
Oct 31 21:44:02.967607 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:02.967575    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj" podUID="1f5fd98d-9122-4102-b317-d51242d31d0e" containerName="istio-proxy" containerID="cri-o://13e50e9fcf11ba1cbb82650bac346c1f8a8780b1eef1b91771b802ef512720ad" gracePeriod=30
Oct 31 21:44:02.986270 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:02.986236    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:03.264024 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:03.258528    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:03.293352 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:03.293314    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc" event={"ID":"19fcad1c-5c6e-4274-936d-bf93ca76c7d4","Type":"ContainerStarted","Data":"9b1af9a77e34f42b623c8e37da558b7140d452b72431ff1c949b7ceb52ec6fff"}
Oct 31 21:44:04.045970 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:04.042275    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:44:04.046981 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:04.046912    4460 patch_prober.go:29] interesting pod/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused" start-of-body=
Oct 31 21:44:04.047119 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:04.047004    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc" podUID="19fcad1c-5c6e-4274-936d-bf93ca76c7d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused"
Oct 31 21:44:04.903216 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:04.903180    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:05.042987 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:05.042841    4460 patch_prober.go:29] interesting pod/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused" start-of-body=
Oct 31 21:44:05.043179 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:05.042987    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc" podUID="19fcad1c-5c6e-4274-936d-bf93ca76c7d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused"
Oct 31 21:44:05.044345 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:05.043963    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:06.037962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.035344    4460 patch_prober.go:29] interesting pod/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused" start-of-body=
Oct 31 21:44:06.037962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.035408    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc" podUID="19fcad1c-5c6e-4274-936d-bf93ca76c7d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused"
Oct 31 21:44:06.365966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.361228    4460 patch_prober.go:29] interesting pod/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 21:44:06.365966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.361290    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj" podUID="1f5fd98d-9122-4102-b317-d51242d31d0e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 21:44:06.583877 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.583791    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:44:06.649505 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.649479    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:44:06.766037 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.766004    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-credential-socket\") pod \"475195a2-42a8-493d-ab68-585d187a977d\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") "
Oct 31 21:44:06.766242 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.766064    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/475195a2-42a8-493d-ab68-585d187a977d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/475195a2-42a8-493d-ab68-585d187a977d-istiod-ca-cert\") pod \"475195a2-42a8-493d-ab68-585d187a977d\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") "
Oct 31 21:44:06.766242 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.766099    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-istio-data\") pod \"475195a2-42a8-493d-ab68-585d187a977d\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") "
Oct 31 21:44:06.766242 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.766164    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-workload-certs\") pod \"475195a2-42a8-493d-ab68-585d187a977d\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") "
Oct 31 21:44:06.766242 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.766221    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/475195a2-42a8-493d-ab68-585d187a977d-kube-api-access-2bpkf\" (UniqueName: \"kubernetes.io/projected/475195a2-42a8-493d-ab68-585d187a977d-kube-api-access-2bpkf\") pod \"475195a2-42a8-493d-ab68-585d187a977d\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") "
Oct 31 21:44:06.766482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.766295    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-credential-socket" pod "475195a2-42a8-493d-ab68-585d187a977d" (UID: "475195a2-42a8-493d-ab68-585d187a977d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:44:06.766482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.766459    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/475195a2-42a8-493d-ab68-585d187a977d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/475195a2-42a8-493d-ab68-585d187a977d-istio-podinfo\") pod \"475195a2-42a8-493d-ab68-585d187a977d\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") "
Oct 31 21:44:06.766601 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.766492    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-workload-socket\") pod \"475195a2-42a8-493d-ab68-585d187a977d\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") "
Oct 31 21:44:06.766601 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.766519    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-istio-envoy\") pod \"475195a2-42a8-493d-ab68-585d187a977d\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") "
Oct 31 21:44:06.766601 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.766534    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-istio-data" pod "475195a2-42a8-493d-ab68-585d187a977d" (UID: "475195a2-42a8-493d-ab68-585d187a977d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:44:06.766601 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.766548    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-workload-certs" pod "475195a2-42a8-493d-ab68-585d187a977d" (UID: "475195a2-42a8-493d-ab68-585d187a977d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:44:06.766601 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.766555    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/475195a2-42a8-493d-ab68-585d187a977d-istio-token\" (UniqueName: \"kubernetes.io/projected/475195a2-42a8-493d-ab68-585d187a977d-istio-token\") pod \"475195a2-42a8-493d-ab68-585d187a977d\" (UID: \"475195a2-42a8-493d-ab68-585d187a977d\") "
Oct 31 21:44:06.766601 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.766569    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/475195a2-42a8-493d-ab68-585d187a977d-istiod-ca-cert" pod "475195a2-42a8-493d-ab68-585d187a977d" (UID: "475195a2-42a8-493d-ab68-585d187a977d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:44:06.766865 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.766739    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-workload-socket" pod "475195a2-42a8-493d-ab68-585d187a977d" (UID: "475195a2-42a8-493d-ab68-585d187a977d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:44:06.767276 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.767253    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:44:06.767441 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.767420    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:44:06.767441 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.767443    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/475195a2-42a8-493d-ab68-585d187a977d-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:44:06.767631 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.767457    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:44:06.767631 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.767471    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:44:06.769119 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.769093    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/475195a2-42a8-493d-ab68-585d187a977d-istio-podinfo" pod "475195a2-42a8-493d-ab68-585d187a977d" (UID: "475195a2-42a8-493d-ab68-585d187a977d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:44:06.769205 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.769101    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/475195a2-42a8-493d-ab68-585d187a977d-kube-api-access-2bpkf" pod "475195a2-42a8-493d-ab68-585d187a977d" (UID: "475195a2-42a8-493d-ab68-585d187a977d"). InnerVolumeSpecName "kube-api-access-2bpkf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:44:06.769252 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.769197    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-istio-envoy" pod "475195a2-42a8-493d-ab68-585d187a977d" (UID: "475195a2-42a8-493d-ab68-585d187a977d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:44:06.769252 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.769207    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/475195a2-42a8-493d-ab68-585d187a977d-istio-token" pod "475195a2-42a8-493d-ab68-585d187a977d" (UID: "475195a2-42a8-493d-ab68-585d187a977d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:44:06.809045 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.809012    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:06.878787 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.878644    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bpkf\" (UniqueName: \"kubernetes.io/projected/475195a2-42a8-493d-ab68-585d187a977d-kube-api-access-2bpkf\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:44:06.878787 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.878684    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/475195a2-42a8-493d-ab68-585d187a977d-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:44:06.878787 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.878697    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/475195a2-42a8-493d-ab68-585d187a977d-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:44:06.878787 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:06.878709    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/475195a2-42a8-493d-ab68-585d187a977d-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:44:07.039381 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:07.039341    4460 patch_prober.go:29] interesting pod/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused" start-of-body=
Oct 31 21:44:07.039977 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:07.039916    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc" podUID="19fcad1c-5c6e-4274-936d-bf93ca76c7d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused"
Oct 31 21:44:07.306201 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:07.306167    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:07.335965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:07.320173    4460 generic.go:356] "Generic (PLEG): container finished" podID="475195a2-42a8-493d-ab68-585d187a977d" containerID="6ce9e89101e01a48e317553d74bcc248326d0601a67cc7885defba4e1fef4d36" exitCode=0
Oct 31 21:44:07.340967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:07.331999    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85" event={"ID":"475195a2-42a8-493d-ab68-585d187a977d","Type":"ContainerDied","Data":"6ce9e89101e01a48e317553d74bcc248326d0601a67cc7885defba4e1fef4d36"}
Oct 31 21:44:07.340967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:07.336275    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85" event={"ID":"475195a2-42a8-493d-ab68-585d187a977d","Type":"ContainerDied","Data":"156e4be1befc4b2fe091268c50abf7afb4a9225969753a2d08ad5bffdd1fe2eb"}
Oct 31 21:44:07.340967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:07.336311    4460 scope.go:122] "RemoveContainer" containerID="6ce9e89101e01a48e317553d74bcc248326d0601a67cc7885defba4e1fef4d36"
Oct 31 21:44:07.340967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:07.332091    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"
Oct 31 21:44:07.354159 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:07.354142    4460 scope.go:122] "RemoveContainer" containerID="6ce9e89101e01a48e317553d74bcc248326d0601a67cc7885defba4e1fef4d36"
Oct 31 21:44:07.355212 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:44:07.355188    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ce9e89101e01a48e317553d74bcc248326d0601a67cc7885defba4e1fef4d36\": container with ID starting with 6ce9e89101e01a48e317553d74bcc248326d0601a67cc7885defba4e1fef4d36 not found: ID does not exist" containerID="6ce9e89101e01a48e317553d74bcc248326d0601a67cc7885defba4e1fef4d36"
Oct 31 21:44:07.355301 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:07.355230    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ce9e89101e01a48e317553d74bcc248326d0601a67cc7885defba4e1fef4d36"} err="failed to get container status \"6ce9e89101e01a48e317553d74bcc248326d0601a67cc7885defba4e1fef4d36\": rpc error: code = NotFound desc = could not find container \"6ce9e89101e01a48e317553d74bcc248326d0601a67cc7885defba4e1fef4d36\": container with ID starting with 6ce9e89101e01a48e317553d74bcc248326d0601a67cc7885defba4e1fef4d36 not found: ID does not exist"
Oct 31 21:44:07.417960 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:07.417913    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"]
Oct 31 21:44:07.418835 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:07.418804    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:07.446737 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:07.446706    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-cqn85"]
Oct 31 21:44:07.673528 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:07.670490    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:08.045319 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.045288    4460 patch_prober.go:29] interesting pod/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused" start-of-body=
Oct 31 21:44:08.045677 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.045342    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc" podUID="19fcad1c-5c6e-4274-936d-bf93ca76c7d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused"
Oct 31 21:44:08.230576 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.230532    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="475195a2-42a8-493d-ab68-585d187a977d" path="/var/lib/kubelet/pods/475195a2-42a8-493d-ab68-585d187a977d/volumes"
Oct 31 21:44:08.284367 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.284336    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:44:08.410963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.408373    4460 generic.go:356] "Generic (PLEG): container finished" podID="1f5fd98d-9122-4102-b317-d51242d31d0e" containerID="13e50e9fcf11ba1cbb82650bac346c1f8a8780b1eef1b91771b802ef512720ad" exitCode=0
Oct 31 21:44:08.410963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.408628    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj" event={"ID":"1f5fd98d-9122-4102-b317-d51242d31d0e","Type":"ContainerDied","Data":"13e50e9fcf11ba1cbb82650bac346c1f8a8780b1eef1b91771b802ef512720ad"}
Oct 31 21:44:08.410963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.408686    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj" event={"ID":"1f5fd98d-9122-4102-b317-d51242d31d0e","Type":"ContainerDied","Data":"89135fe4b0de02fa7319b2869757eff3f6ee57248c37d22abf1444ecfb2cf72a"}
Oct 31 21:44:08.410963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.408717    4460 scope.go:122] "RemoveContainer" containerID="13e50e9fcf11ba1cbb82650bac346c1f8a8780b1eef1b91771b802ef512720ad"
Oct 31 21:44:08.410963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.408869    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"
Oct 31 21:44:08.443282 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.429438    4460 scope.go:122] "RemoveContainer" containerID="d0afd41ea2de72627ed621723c6cc15a3e378594eb25ebfe0cbe61d6cffaee12"
Oct 31 21:44:08.455971 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.455601    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-istio-envoy\") pod \"1f5fd98d-9122-4102-b317-d51242d31d0e\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") "
Oct 31 21:44:08.455971 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.455691    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f5fd98d-9122-4102-b317-d51242d31d0e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f5fd98d-9122-4102-b317-d51242d31d0e-istiod-ca-cert\") pod \"1f5fd98d-9122-4102-b317-d51242d31d0e\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") "
Oct 31 21:44:08.455971 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.455726    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1f5fd98d-9122-4102-b317-d51242d31d0e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f5fd98d-9122-4102-b317-d51242d31d0e-istio-podinfo\") pod \"1f5fd98d-9122-4102-b317-d51242d31d0e\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") "
Oct 31 21:44:08.455971 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.455771    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-workload-certs\") pod \"1f5fd98d-9122-4102-b317-d51242d31d0e\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") "
Oct 31 21:44:08.455971 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.455832    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-workload-socket\") pod \"1f5fd98d-9122-4102-b317-d51242d31d0e\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") "
Oct 31 21:44:08.456370 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.455933    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-credential-socket\") pod \"1f5fd98d-9122-4102-b317-d51242d31d0e\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") "
Oct 31 21:44:08.456370 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.456065    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f5fd98d-9122-4102-b317-d51242d31d0e-istio-token\" (UniqueName: \"kubernetes.io/projected/1f5fd98d-9122-4102-b317-d51242d31d0e-istio-token\") pod \"1f5fd98d-9122-4102-b317-d51242d31d0e\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") "
Oct 31 21:44:08.456370 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.456137    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f5fd98d-9122-4102-b317-d51242d31d0e-kube-api-access-9qxs7\" (UniqueName: \"kubernetes.io/projected/1f5fd98d-9122-4102-b317-d51242d31d0e-kube-api-access-9qxs7\") pod \"1f5fd98d-9122-4102-b317-d51242d31d0e\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") "
Oct 31 21:44:08.456370 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.456243    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-istio-data\") pod \"1f5fd98d-9122-4102-b317-d51242d31d0e\" (UID: \"1f5fd98d-9122-4102-b317-d51242d31d0e\") "
Oct 31 21:44:08.473999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.457186    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-istio-data" pod "1f5fd98d-9122-4102-b317-d51242d31d0e" (UID: "1f5fd98d-9122-4102-b317-d51242d31d0e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:44:08.473999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.460216    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-workload-socket" pod "1f5fd98d-9122-4102-b317-d51242d31d0e" (UID: "1f5fd98d-9122-4102-b317-d51242d31d0e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:44:08.473999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.460548    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f5fd98d-9122-4102-b317-d51242d31d0e-istiod-ca-cert" pod "1f5fd98d-9122-4102-b317-d51242d31d0e" (UID: "1f5fd98d-9122-4102-b317-d51242d31d0e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:44:08.473999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.464185    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-istio-envoy" pod "1f5fd98d-9122-4102-b317-d51242d31d0e" (UID: "1f5fd98d-9122-4102-b317-d51242d31d0e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:44:08.473999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.464375    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-credential-socket" pod "1f5fd98d-9122-4102-b317-d51242d31d0e" (UID: "1f5fd98d-9122-4102-b317-d51242d31d0e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:44:08.473999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.465223    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-workload-certs" pod "1f5fd98d-9122-4102-b317-d51242d31d0e" (UID: "1f5fd98d-9122-4102-b317-d51242d31d0e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:44:08.473999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.466124    4460 scope.go:122] "RemoveContainer" containerID="13e50e9fcf11ba1cbb82650bac346c1f8a8780b1eef1b91771b802ef512720ad"
Oct 31 21:44:08.473999 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:44:08.466712    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13e50e9fcf11ba1cbb82650bac346c1f8a8780b1eef1b91771b802ef512720ad\": container with ID starting with 13e50e9fcf11ba1cbb82650bac346c1f8a8780b1eef1b91771b802ef512720ad not found: ID does not exist" containerID="13e50e9fcf11ba1cbb82650bac346c1f8a8780b1eef1b91771b802ef512720ad"
Oct 31 21:44:08.473999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.466859    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13e50e9fcf11ba1cbb82650bac346c1f8a8780b1eef1b91771b802ef512720ad"} err="failed to get container status \"13e50e9fcf11ba1cbb82650bac346c1f8a8780b1eef1b91771b802ef512720ad\": rpc error: code = NotFound desc = could not find container \"13e50e9fcf11ba1cbb82650bac346c1f8a8780b1eef1b91771b802ef512720ad\": container with ID starting with 13e50e9fcf11ba1cbb82650bac346c1f8a8780b1eef1b91771b802ef512720ad not found: ID does not exist"
Oct 31 21:44:08.473999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.466892    4460 scope.go:122] "RemoveContainer" containerID="d0afd41ea2de72627ed621723c6cc15a3e378594eb25ebfe0cbe61d6cffaee12"
Oct 31 21:44:08.473999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.468258    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1f5fd98d-9122-4102-b317-d51242d31d0e-istio-podinfo" pod "1f5fd98d-9122-4102-b317-d51242d31d0e" (UID: "1f5fd98d-9122-4102-b317-d51242d31d0e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:44:08.473999 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:44:08.468361    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0afd41ea2de72627ed621723c6cc15a3e378594eb25ebfe0cbe61d6cffaee12\": container with ID starting with d0afd41ea2de72627ed621723c6cc15a3e378594eb25ebfe0cbe61d6cffaee12 not found: ID does not exist" containerID="d0afd41ea2de72627ed621723c6cc15a3e378594eb25ebfe0cbe61d6cffaee12"
Oct 31 21:44:08.473999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.468392    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0afd41ea2de72627ed621723c6cc15a3e378594eb25ebfe0cbe61d6cffaee12"} err="failed to get container status \"d0afd41ea2de72627ed621723c6cc15a3e378594eb25ebfe0cbe61d6cffaee12\": rpc error: code = NotFound desc = could not find container \"d0afd41ea2de72627ed621723c6cc15a3e378594eb25ebfe0cbe61d6cffaee12\": container with ID starting with d0afd41ea2de72627ed621723c6cc15a3e378594eb25ebfe0cbe61d6cffaee12 not found: ID does not exist"
Oct 31 21:44:08.473999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.468880    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f5fd98d-9122-4102-b317-d51242d31d0e-istio-token" pod "1f5fd98d-9122-4102-b317-d51242d31d0e" (UID: "1f5fd98d-9122-4102-b317-d51242d31d0e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:44:08.483873 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.482179    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f5fd98d-9122-4102-b317-d51242d31d0e-kube-api-access-9qxs7" pod "1f5fd98d-9122-4102-b317-d51242d31d0e" (UID: "1f5fd98d-9122-4102-b317-d51242d31d0e"). InnerVolumeSpecName "kube-api-access-9qxs7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:44:08.558103 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.558071    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:44:08.558103 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.558104    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f5fd98d-9122-4102-b317-d51242d31d0e-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:44:08.558350 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.558114    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f5fd98d-9122-4102-b317-d51242d31d0e-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:44:08.558350 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.558122    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:44:08.558350 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.558132    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:44:08.558350 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.558141    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:44:08.558350 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.558150    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f5fd98d-9122-4102-b317-d51242d31d0e-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:44:08.558350 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.558159    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qxs7\" (UniqueName: \"kubernetes.io/projected/1f5fd98d-9122-4102-b317-d51242d31d0e-kube-api-access-9qxs7\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:44:08.558350 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.558172    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f5fd98d-9122-4102-b317-d51242d31d0e-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:44:08.692848 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.692819    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:08.789966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.788993    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"]
Oct 31 21:44:08.795498 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:08.795455    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-a-6d6576-openshift-default-6b8bdd8dbf-5q4qj"]
Oct 31 21:44:09.036208 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.036179    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:44:09.037162 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.037141    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:44:09.208269 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.208238    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:09.332057 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.331975    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:09.679266 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.679171    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"]
Oct 31 21:44:09.680194 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.680167    4460 cpu_manager.go:414] "containerMap: removing container" podUID="7e7288b0-58f2-41e8-a26e-eddac51598e0" containerName="istio-proxy"
Oct 31 21:44:09.680194 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.680194    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7e7288b0-58f2-41e8-a26e-eddac51598e0" containerName="istio-proxy"
Oct 31 21:44:09.680369 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.680214    4460 cpu_manager.go:414] "containerMap: removing container" podUID="475195a2-42a8-493d-ab68-585d187a977d" containerName="istio-proxy"
Oct 31 21:44:09.680369 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.680222    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="475195a2-42a8-493d-ab68-585d187a977d" containerName="istio-proxy"
Oct 31 21:44:09.680369 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.680239    4460 cpu_manager.go:414] "containerMap: removing container" podUID="692f0926-1719-44a1-94a9-12604a64cf40" containerName="istio-proxy"
Oct 31 21:44:09.680369 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.680249    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="692f0926-1719-44a1-94a9-12604a64cf40" containerName="istio-proxy"
Oct 31 21:44:09.680369 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.680281    4460 cpu_manager.go:414] "containerMap: removing container" podUID="1f5fd98d-9122-4102-b317-d51242d31d0e" containerName="istio-proxy"
Oct 31 21:44:09.680369 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.680289    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f5fd98d-9122-4102-b317-d51242d31d0e" containerName="istio-proxy"
Oct 31 21:44:09.680369 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.680318    4460 cpu_manager.go:414] "containerMap: removing container" podUID="1f5fd98d-9122-4102-b317-d51242d31d0e" containerName="istio-proxy"
Oct 31 21:44:09.680369 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.680325    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f5fd98d-9122-4102-b317-d51242d31d0e" containerName="istio-proxy"
Oct 31 21:44:09.680599 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.680478    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="7e7288b0-58f2-41e8-a26e-eddac51598e0" containerName="istio-proxy"
Oct 31 21:44:09.680599 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.680494    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="475195a2-42a8-493d-ab68-585d187a977d" containerName="istio-proxy"
Oct 31 21:44:09.680599 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.680505    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="692f0926-1719-44a1-94a9-12604a64cf40" containerName="istio-proxy"
Oct 31 21:44:09.680599 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.680517    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f5fd98d-9122-4102-b317-d51242d31d0e" containerName="istio-proxy"
Oct 31 21:44:09.680599 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.680528    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f5fd98d-9122-4102-b317-d51242d31d0e" containerName="istio-proxy"
Oct 31 21:44:09.686023 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.686002    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:44:09.704301 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.704273    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-istio-data\") pod \"e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") " pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:44:09.704486 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.704465    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78bb7115-5485-45ec-9b35-a76a75ba3eff-istio-token\") pod \"e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") " pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:44:09.704576 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.704527    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-istio-envoy\") pod \"e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") " pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:44:09.704637 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.704573    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-workload-socket\") pod \"e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") " pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:44:09.704637 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.704599    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-workload-certs\") pod \"e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") " pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:44:09.704637 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.704624    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtj9g\" (UniqueName: \"kubernetes.io/projected/78bb7115-5485-45ec-9b35-a76a75ba3eff-kube-api-access-xtj9g\") pod \"e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") " pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:44:09.704790 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.704687    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78bb7115-5485-45ec-9b35-a76a75ba3eff-istio-podinfo\") pod \"e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") " pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:44:09.704790 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.704731    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78bb7115-5485-45ec-9b35-a76a75ba3eff-istiod-ca-cert\") pod \"e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") " pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:44:09.704887 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.704841    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-credential-socket\") pod \"e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") " pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:44:09.707623 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.707598    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"]
Oct 31 21:44:09.715927 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.715907    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-a-448b01-openshift-default-dockercfg-czclc\""
Oct 31 21:44:09.726524 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.726488    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:09.815230 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.815187    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78bb7115-5485-45ec-9b35-a76a75ba3eff-istio-token\") pod \"e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") " pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:44:09.827144 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.827107    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-istio-envoy\") pod \"e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") " pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:44:09.827413 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.827393    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-workload-socket\") pod \"e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") " pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:44:09.827538 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.827522    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-workload-certs\") pod \"e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") " pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:44:09.850964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.828106    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-workload-certs\") pod \"e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") " pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:44:09.851215 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.830013    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtj9g\" (UniqueName: \"kubernetes.io/projected/78bb7115-5485-45ec-9b35-a76a75ba3eff-kube-api-access-xtj9g\") pod \"e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") " pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:44:09.851443 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.851419    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78bb7115-5485-45ec-9b35-a76a75ba3eff-istio-podinfo\") pod \"e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") " pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:44:09.851595 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.851580    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78bb7115-5485-45ec-9b35-a76a75ba3eff-istiod-ca-cert\") pod \"e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") " pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:44:09.851815 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.851796    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-credential-socket\") pod \"e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") " pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:44:09.852043 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.852019    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-istio-data\") pod \"e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") " pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:44:09.874961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.852615    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-istio-data\") pod \"e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") " pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:44:09.875206 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.836563    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-istio-envoy\") pod \"e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") " pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:44:09.875334 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.830957    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-workload-socket\") pod \"e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") " pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:44:09.875457 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.853621    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78bb7115-5485-45ec-9b35-a76a75ba3eff-istiod-ca-cert\") pod \"e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") " pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:44:09.875566 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.853852    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-credential-socket\") pod \"e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") " pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:44:09.875670 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.870895    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78bb7115-5485-45ec-9b35-a76a75ba3eff-istio-podinfo\") pod \"e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") " pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:44:09.886012 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.885676    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtj9g\" (UniqueName: \"kubernetes.io/projected/78bb7115-5485-45ec-9b35-a76a75ba3eff-kube-api-access-xtj9g\") pod \"e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") " pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:44:09.892964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:09.890422    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78bb7115-5485-45ec-9b35-a76a75ba3eff-istio-token\") pod \"e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") " pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:44:10.000549 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:10.000457    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:44:10.080977 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:10.080325    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:10.227421 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:10.227384    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1f5fd98d-9122-4102-b317-d51242d31d0e" path="/var/lib/kubelet/pods/1f5fd98d-9122-4102-b317-d51242d31d0e/volumes"
Oct 31 21:44:10.322978 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:44:10.322909    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78bb7115_5485_45ec_9b35_a76a75ba3eff.slice/crio-4d8f2f5a3890ebb7c2cccc4d7aaf633c2df7e7fea0ccf40b9d63e72917f3a3f5 WatchSource:0}: Error finding container 4d8f2f5a3890ebb7c2cccc4d7aaf633c2df7e7fea0ccf40b9d63e72917f3a3f5: Status 404 returned error can't find the container with id 4d8f2f5a3890ebb7c2cccc4d7aaf633c2df7e7fea0ccf40b9d63e72917f3a3f5
Oct 31 21:44:10.323494 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:10.323468    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"]
Oct 31 21:44:10.326610 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:10.326568    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:44:10.326704 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:10.326672    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:44:10.326764 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:10.326732    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:44:10.453411 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:10.453213    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns" event={"ID":"78bb7115-5485-45ec-9b35-a76a75ba3eff","Type":"ContainerStarted","Data":"9b78df3386149b2efadc188f14b892e0d594f5cd5141821fc388f25309f65331"}
Oct 31 21:44:10.453411 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:10.453331    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns" event={"ID":"78bb7115-5485-45ec-9b35-a76a75ba3eff","Type":"ContainerStarted","Data":"4d8f2f5a3890ebb7c2cccc4d7aaf633c2df7e7fea0ccf40b9d63e72917f3a3f5"}
Oct 31 21:44:10.503203 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:10.503148    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns" podStartSLOduration=1.503135093 podStartE2EDuration="1.503135093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:44:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:44:10.501714498 +0000 UTC m=+1690.418023929" watchObservedRunningTime="2026-10-31 21:44:10.503135093 +0000 UTC m=+1690.419444583"
Oct 31 21:44:10.615464 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:10.615372    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:11.001244 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:11.001170    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:44:11.006893 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:11.006863    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:44:11.461672 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:11.461640    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:44:11.462854 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:11.462833    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:44:12.687977 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:12.687921    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:12.936027 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:12.935978    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:13.050536 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:13.050493    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:14.631407 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:14.631369    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:14.743649 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:14.743614    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:14.813698 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:14.813664    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-448b01-67f46999c5-jlp47"]
Oct 31 21:44:14.818556 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:14.818538    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-448b01-67f46999c5-jlp47"
Oct 31 21:44:14.836967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:14.836926    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-448b01-67f46999c5-jlp47"]
Oct 31 21:44:14.847030 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:14.847005    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-a-448b01-dockercfg-4zg7h\""
Oct 31 21:44:14.847423 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:14.847400    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-a-448b01\""
Oct 31 21:44:14.953009 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:14.942544    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bad71d55-48bf-4120-affb-c627a9988e3e-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-448b01-67f46999c5-jlp47\" (UID: \"bad71d55-48bf-4120-affb-c627a9988e3e\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-448b01-67f46999c5-jlp47"
Oct 31 21:44:14.953293 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:14.953264    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsr4w\" (UniqueName: \"kubernetes.io/projected/bad71d55-48bf-4120-affb-c627a9988e3e-kube-api-access-nsr4w\") pod \"payload-pre-processing-e2e-praxis-a-448b01-67f46999c5-jlp47\" (UID: \"bad71d55-48bf-4120-affb-c627a9988e3e\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-448b01-67f46999c5-jlp47"
Oct 31 21:44:15.017042 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:15.017001    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-448b01-6778f95d9-mjvcz"]
Oct 31 21:44:15.061964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:15.058068    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bad71d55-48bf-4120-affb-c627a9988e3e-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-448b01-67f46999c5-jlp47\" (UID: \"bad71d55-48bf-4120-affb-c627a9988e3e\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-448b01-67f46999c5-jlp47"
Oct 31 21:44:15.061964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:15.058118    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsr4w\" (UniqueName: \"kubernetes.io/projected/bad71d55-48bf-4120-affb-c627a9988e3e-kube-api-access-nsr4w\") pod \"payload-pre-processing-e2e-praxis-a-448b01-67f46999c5-jlp47\" (UID: \"bad71d55-48bf-4120-affb-c627a9988e3e\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-448b01-67f46999c5-jlp47"
Oct 31 21:44:15.061964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:15.059512    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bad71d55-48bf-4120-affb-c627a9988e3e-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-448b01-67f46999c5-jlp47\" (UID: \"bad71d55-48bf-4120-affb-c627a9988e3e\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-448b01-67f46999c5-jlp47"
Oct 31 21:44:15.062319 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:15.062253    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-448b01-6778f95d9-mjvcz"
Oct 31 21:44:15.065972 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:15.065881    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-448b01-6778f95d9-mjvcz"]
Oct 31 21:44:15.108777 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:15.108683    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsr4w\" (UniqueName: \"kubernetes.io/projected/bad71d55-48bf-4120-affb-c627a9988e3e-kube-api-access-nsr4w\") pod \"payload-pre-processing-e2e-praxis-a-448b01-67f46999c5-jlp47\" (UID: \"bad71d55-48bf-4120-affb-c627a9988e3e\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-448b01-67f46999c5-jlp47"
Oct 31 21:44:15.134460 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:15.134432    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-448b01-67f46999c5-jlp47"
Oct 31 21:44:15.160314 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:15.160274    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2113003b-165f-4533-aff5-85c75433f325-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-448b01-6778f95d9-mjvcz\" (UID: \"2113003b-165f-4533-aff5-85c75433f325\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-448b01-6778f95d9-mjvcz"
Oct 31 21:44:15.160451 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:15.160353    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9t9t\" (UniqueName: \"kubernetes.io/projected/2113003b-165f-4533-aff5-85c75433f325-kube-api-access-f9t9t\") pod \"payload-processing-e2e-praxis-a-448b01-6778f95d9-mjvcz\" (UID: \"2113003b-165f-4533-aff5-85c75433f325\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-448b01-6778f95d9-mjvcz"
Oct 31 21:44:15.259910 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:15.258120    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:15.262579 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:15.262546    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2113003b-165f-4533-aff5-85c75433f325-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-448b01-6778f95d9-mjvcz\" (UID: \"2113003b-165f-4533-aff5-85c75433f325\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-448b01-6778f95d9-mjvcz"
Oct 31 21:44:15.262729 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:15.262630    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9t9t\" (UniqueName: \"kubernetes.io/projected/2113003b-165f-4533-aff5-85c75433f325-kube-api-access-f9t9t\") pod \"payload-processing-e2e-praxis-a-448b01-6778f95d9-mjvcz\" (UID: \"2113003b-165f-4533-aff5-85c75433f325\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-448b01-6778f95d9-mjvcz"
Oct 31 21:44:15.263926 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:15.263902    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2113003b-165f-4533-aff5-85c75433f325-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-448b01-6778f95d9-mjvcz\" (UID: \"2113003b-165f-4533-aff5-85c75433f325\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-448b01-6778f95d9-mjvcz"
Oct 31 21:44:15.287656 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:15.287620    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9t9t\" (UniqueName: \"kubernetes.io/projected/2113003b-165f-4533-aff5-85c75433f325-kube-api-access-f9t9t\") pod \"payload-processing-e2e-praxis-a-448b01-6778f95d9-mjvcz\" (UID: \"2113003b-165f-4533-aff5-85c75433f325\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-448b01-6778f95d9-mjvcz"
Oct 31 21:44:15.387228 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:15.387198    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-448b01-6778f95d9-mjvcz"
Oct 31 21:44:15.449398 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:15.449370    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-448b01-67f46999c5-jlp47"]
Oct 31 21:44:15.494149 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:15.493112    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-448b01-67f46999c5-jlp47" event={"ID":"bad71d55-48bf-4120-affb-c627a9988e3e","Type":"ContainerStarted","Data":"e1ddaebae4be7c3e05ad595b1e50a4aeaedfac2cf2a719aca59048164dcebf6d"}
Oct 31 21:44:15.697483 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:15.697450    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-448b01-6778f95d9-mjvcz"]
Oct 31 21:44:15.697855 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:44:15.697829    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2113003b_165f_4533_aff5_85c75433f325.slice/crio-b0854fb4ac31e27062bfcd077ef153bd9ea15d80158bf0d55a8c9b030b903c42 WatchSource:0}: Error finding container b0854fb4ac31e27062bfcd077ef153bd9ea15d80158bf0d55a8c9b030b903c42: Status 404 returned error can't find the container with id b0854fb4ac31e27062bfcd077ef153bd9ea15d80158bf0d55a8c9b030b903c42
Oct 31 21:44:16.501206 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:16.501170    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-448b01-67f46999c5-jlp47" event={"ID":"bad71d55-48bf-4120-affb-c627a9988e3e","Type":"ContainerStarted","Data":"ee62bfe19404a037801a04e1a156dbb87eb539ad020d40fda901089063eb7b02"}
Oct 31 21:44:16.501409 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:16.501291    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-448b01-67f46999c5-jlp47"
Oct 31 21:44:16.503041 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:16.503015    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-448b01-6778f95d9-mjvcz" event={"ID":"2113003b-165f-4533-aff5-85c75433f325","Type":"ContainerStarted","Data":"2cc74fc21290b71d3c1aac8a2c8cfdfcb0e4dc149df3babdf8fd9f5be0971374"}
Oct 31 21:44:16.503161 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:16.503056    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-448b01-6778f95d9-mjvcz" event={"ID":"2113003b-165f-4533-aff5-85c75433f325","Type":"ContainerStarted","Data":"b0854fb4ac31e27062bfcd077ef153bd9ea15d80158bf0d55a8c9b030b903c42"}
Oct 31 21:44:16.503161 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:16.503076    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-448b01-6778f95d9-mjvcz"
Oct 31 21:44:16.547746 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:16.547701    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-448b01-67f46999c5-jlp47" podStartSLOduration=2.547687395 podStartE2EDuration="2.547687395s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:44:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:44:16.544451867 +0000 UTC m=+1696.460761295" watchObservedRunningTime="2026-10-31 21:44:16.547687395 +0000 UTC m=+1696.463996823"
Oct 31 21:44:16.590512 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:16.590461    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-praxis-a-448b01-6778f95d9-mjvcz" podStartSLOduration=2.590447713 podStartE2EDuration="2.590447713s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:44:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:44:16.58810668 +0000 UTC m=+1696.504416111" watchObservedRunningTime="2026-10-31 21:44:16.590447713 +0000 UTC m=+1696.506757141"
Oct 31 21:44:17.029384 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:17.029351    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:17.824354 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:17.824314    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:44:18.843963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:18.843385    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:19.043966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:19.043812    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-bb9990-644f4468f9-tvk5r"]
Oct 31 21:44:19.170302 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:19.170101    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:19.291231 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:19.291196    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:20.546829 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:20.546782    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:21.470926 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:44:21.470886    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:44:22.252668 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:22.252622    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:22.400243 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:22.399614    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:22.529646 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:22.529609    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:24.044395 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:24.044329    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-praxis-b-bb9990-644f4468f9-tvk5r" podUID="4dc613a9-a4d1-437d-87df-9c13daa4bb05" containerName="payload-processing" containerID="cri-o://bf601a0b86d6909c149a640bb5b23bfccb34404418aa9e9221b8187da83b981b" gracePeriod=25
Oct 31 21:44:24.141348 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:24.141316    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:24.378830 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:24.378796    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-bb9990-644f4468f9-tvk5r"
Oct 31 21:44:24.472241 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:24.472204    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4dc613a9-a4d1-437d-87df-9c13daa4bb05-kube-api-access-qqp5k\" (UniqueName: \"kubernetes.io/projected/4dc613a9-a4d1-437d-87df-9c13daa4bb05-kube-api-access-qqp5k\") pod \"4dc613a9-a4d1-437d-87df-9c13daa4bb05\" (UID: \"4dc613a9-a4d1-437d-87df-9c13daa4bb05\") "
Oct 31 21:44:24.472463 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:24.472443    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4dc613a9-a4d1-437d-87df-9c13daa4bb05-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dc613a9-a4d1-437d-87df-9c13daa4bb05-plugins-config-volume\") pod \"4dc613a9-a4d1-437d-87df-9c13daa4bb05\" (UID: \"4dc613a9-a4d1-437d-87df-9c13daa4bb05\") "
Oct 31 21:44:24.472912 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:24.472878    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4dc613a9-a4d1-437d-87df-9c13daa4bb05-plugins-config-volume" pod "4dc613a9-a4d1-437d-87df-9c13daa4bb05" (UID: "4dc613a9-a4d1-437d-87df-9c13daa4bb05"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:44:24.473058 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:24.473023    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4dc613a9-a4d1-437d-87df-9c13daa4bb05-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:44:24.475147 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:24.475118    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dc613a9-a4d1-437d-87df-9c13daa4bb05-kube-api-access-qqp5k" pod "4dc613a9-a4d1-437d-87df-9c13daa4bb05" (UID: "4dc613a9-a4d1-437d-87df-9c13daa4bb05"). InnerVolumeSpecName "kube-api-access-qqp5k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:44:24.574972 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:24.574901    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqp5k\" (UniqueName: \"kubernetes.io/projected/4dc613a9-a4d1-437d-87df-9c13daa4bb05-kube-api-access-qqp5k\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:44:24.610441 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:24.610343    4460 generic.go:356] "Generic (PLEG): container finished" podID="4dc613a9-a4d1-437d-87df-9c13daa4bb05" containerID="bf601a0b86d6909c149a640bb5b23bfccb34404418aa9e9221b8187da83b981b" exitCode=0
Oct 31 21:44:24.610441 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:24.610398    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-bb9990-644f4468f9-tvk5r"
Oct 31 21:44:24.610441 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:24.610429    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-bb9990-644f4468f9-tvk5r" event={"ID":"4dc613a9-a4d1-437d-87df-9c13daa4bb05","Type":"ContainerDied","Data":"bf601a0b86d6909c149a640bb5b23bfccb34404418aa9e9221b8187da83b981b"}
Oct 31 21:44:24.610724 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:24.610493    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-bb9990-644f4468f9-tvk5r" event={"ID":"4dc613a9-a4d1-437d-87df-9c13daa4bb05","Type":"ContainerDied","Data":"2c536ea0e1a111ebc7b0941054c32d6c722889351b7f2dca0606604ac1b0a4fb"}
Oct 31 21:44:24.610724 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:24.610527    4460 scope.go:122] "RemoveContainer" containerID="bf601a0b86d6909c149a640bb5b23bfccb34404418aa9e9221b8187da83b981b"
Oct 31 21:44:24.621030 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:24.621012    4460 scope.go:122] "RemoveContainer" containerID="bf601a0b86d6909c149a640bb5b23bfccb34404418aa9e9221b8187da83b981b"
Oct 31 21:44:24.621293 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:44:24.621273    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf601a0b86d6909c149a640bb5b23bfccb34404418aa9e9221b8187da83b981b\": container with ID starting with bf601a0b86d6909c149a640bb5b23bfccb34404418aa9e9221b8187da83b981b not found: ID does not exist" containerID="bf601a0b86d6909c149a640bb5b23bfccb34404418aa9e9221b8187da83b981b"
Oct 31 21:44:24.621349 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:24.621302    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf601a0b86d6909c149a640bb5b23bfccb34404418aa9e9221b8187da83b981b"} err="failed to get container status \"bf601a0b86d6909c149a640bb5b23bfccb34404418aa9e9221b8187da83b981b\": rpc error: code = NotFound desc = could not find container \"bf601a0b86d6909c149a640bb5b23bfccb34404418aa9e9221b8187da83b981b\": container with ID starting with bf601a0b86d6909c149a640bb5b23bfccb34404418aa9e9221b8187da83b981b not found: ID does not exist"
Oct 31 21:44:24.668332 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:24.668300    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-bb9990-644f4468f9-tvk5r"]
Oct 31 21:44:24.672233 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:24.672211    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-bb9990-644f4468f9-tvk5r"]
Oct 31 21:44:24.786651 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:24.786616    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:25.815190 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:25.815144    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:26.226726 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:26.226641    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4dc613a9-a4d1-437d-87df-9c13daa4bb05" path="/var/lib/kubelet/pods/4dc613a9-a4d1-437d-87df-9c13daa4bb05/volumes"
Oct 31 21:44:27.386484 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:27.386441    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:27.507486 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:27.507450    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:27.516344 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:27.516311    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-448b01-67f46999c5-jlp47"
Oct 31 21:44:27.518557 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:27.518535    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-448b01-6778f95d9-mjvcz"
Oct 31 21:44:28.646656 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:28.646624    4460 generic.go:356] "Generic (PLEG): container finished" podID="4e8f28aa-6c57-4882-92bf-b1c5f9450da5" containerID="a0dd4795966350e151c1d7302266d13aa94c85669f67dcfb49fe143dab683cd8" exitCode=0
Oct 31 21:44:28.647103 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:28.646673    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj" event={"ID":"4e8f28aa-6c57-4882-92bf-b1c5f9450da5","Type":"ContainerDied","Data":"a0dd4795966350e151c1d7302266d13aa94c85669f67dcfb49fe143dab683cd8"}
Oct 31 21:44:28.647103 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:28.647084    4460 scope.go:122] "RemoveContainer" containerID="a0dd4795966350e151c1d7302266d13aa94c85669f67dcfb49fe143dab683cd8"
Oct 31 21:44:28.648713 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:28.648686    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:44:28.648829 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:28.648750    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:44:28.648829 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:28.648785    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:44:28.803988 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:28.803897    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:28.916437 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:28.916354    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:29.385002 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:29.379480    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:29.494849 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:29.494814    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:29.612970 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:29.608888    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:29.678169 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:29.678084    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj" event={"ID":"4e8f28aa-6c57-4882-92bf-b1c5f9450da5","Type":"ContainerStarted","Data":"07ba5963e1b9608123d9bc8f8c21d0e4b01d1a6131fa03b8e8617170e5d071c8"}
Oct 31 21:44:29.763020 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:29.762985    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:44:29.764379 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:29.764350    4460 patch_prober.go:29] interesting pod/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused" start-of-body=
Oct 31 21:44:29.764515 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:29.764398    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj" podUID="4e8f28aa-6c57-4882-92bf-b1c5f9450da5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused"
Oct 31 21:44:29.983679 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:29.983595    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:30.762774 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:30.762734    4460 patch_prober.go:29] interesting pod/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused" start-of-body=
Oct 31 21:44:30.763176 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:30.762798    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj" podUID="4e8f28aa-6c57-4882-92bf-b1c5f9450da5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused"
Oct 31 21:44:31.451979 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:31.451927    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:31.762572 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:31.762540    4460 patch_prober.go:29] interesting pod/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused" start-of-body=
Oct 31 21:44:31.762725 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:31.762601    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj" podUID="4e8f28aa-6c57-4882-92bf-b1c5f9450da5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused"
Oct 31 21:44:31.800656 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:31.800630    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:44:32.762558 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:32.762520    4460 patch_prober.go:29] interesting pod/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused" start-of-body=
Oct 31 21:44:32.762742 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:32.762573    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj" podUID="4e8f28aa-6c57-4882-92bf-b1c5f9450da5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused"
Oct 31 21:44:32.849533 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:32.849497    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:32.959043 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:32.959004    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:33.291499 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:33.291462    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:33.763076 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:33.763042    4460 patch_prober.go:29] interesting pod/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused" start-of-body=
Oct 31 21:44:33.763319 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:33.763178    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj" podUID="4e8f28aa-6c57-4882-92bf-b1c5f9450da5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused"
Oct 31 21:44:34.762355 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:34.762320    4460 patch_prober.go:29] interesting pod/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused" start-of-body=
Oct 31 21:44:34.762754 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:34.762395    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj" podUID="4e8f28aa-6c57-4882-92bf-b1c5f9450da5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.126:15021/healthz/ready\": dial tcp 10.133.0.126:15021: connect: connection refused"
Oct 31 21:44:35.126067 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:35.125992    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:35.767741 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:35.767709    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:44:35.769053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:35.769029    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:44:36.349053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:36.349019    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:36.481022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:36.480984    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:36.699299 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:36.699212    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:36.813246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:36.813216    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:37.542441 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:37.542383    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:37.664991 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:37.661984    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:37.773750 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:37.773717    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:38.533172 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:38.533137    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:38.654075 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:38.654044    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:38.768112 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:38.768076    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:40.267467 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:40.267433    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:42.278110 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:42.278071    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:44:42.620612 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:42.620532    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:44.413201 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:44.413167    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:44.589699 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:44.589661    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:46.022024 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:46.021988    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:47.005400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:47.005363    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:47.416008 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:47.415931    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:47.910036 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:47.910001    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:48.018834 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:48.018792    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:48.532120 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:48.532084    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:49.697604 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:49.697571    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:51.373058 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:51.373022    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:51.496848 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:44:51.496809    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:44:53.205448 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:53.205413    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:54.664463 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:54.664425    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:54.790707 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:54.790671    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:54.915797 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:54.915716    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:55.003683 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:55.003651    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"]
Oct 31 21:44:55.003973 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:55.003931    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9" podUID="3788ae03-d501-4b1f-89e2-c47c533ead04" containerName="istio-proxy" containerID="cri-o://2300bad6f458d416ac41c8241319d446675d7bae06d11947436c715898284d6c" gracePeriod=30
Oct 31 21:44:55.029854 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:55.029827    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:56.809104 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:56.809058    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:58.115447 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:58.115411    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:58.344491 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:58.344455    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:44:59.384273 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:44:59.384244    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:00.177566 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.177537    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq"]
Oct 31 21:45:00.179769 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.179745    4460 cpu_manager.go:414] "containerMap: removing container" podUID="4dc613a9-a4d1-437d-87df-9c13daa4bb05" containerName="payload-processing"
Oct 31 21:45:00.179980 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.179965    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4dc613a9-a4d1-437d-87df-9c13daa4bb05" containerName="payload-processing"
Oct 31 21:45:00.180219 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.180206    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="4dc613a9-a4d1-437d-87df-9c13daa4bb05" containerName="payload-processing"
Oct 31 21:45:00.193328 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.193299    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq"
Oct 31 21:45:00.198064 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.198032    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shn5n\" (UniqueName: \"kubernetes.io/projected/575790f7-e8f3-49bb-8d95-60f745bc8cff-kube-api-access-shn5n\") pod \"maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq\" (UID: \"575790f7-e8f3-49bb-8d95-60f745bc8cff\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq"
Oct 31 21:45:00.198186 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.198115    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/575790f7-e8f3-49bb-8d95-60f745bc8cff-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq\" (UID: \"575790f7-e8f3-49bb-8d95-60f745bc8cff\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq"
Oct 31 21:45:00.199073 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.199050    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh"]
Oct 31 21:45:00.290775 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.290688    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq"]
Oct 31 21:45:00.292188 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.292162    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh"
Oct 31 21:45:00.304276 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.304243    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shn5n\" (UniqueName: \"kubernetes.io/projected/575790f7-e8f3-49bb-8d95-60f745bc8cff-kube-api-access-shn5n\") pod \"maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq\" (UID: \"575790f7-e8f3-49bb-8d95-60f745bc8cff\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq"
Oct 31 21:45:00.304816 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.304794    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/575790f7-e8f3-49bb-8d95-60f745bc8cff-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq\" (UID: \"575790f7-e8f3-49bb-8d95-60f745bc8cff\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq"
Oct 31 21:45:00.305066 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.305048    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28c4p\" (UniqueName: \"kubernetes.io/projected/9973b2ec-4365-42d4-98c2-f10cea058b4d-kube-api-access-28c4p\") pod \"maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh\" (UID: \"9973b2ec-4365-42d4-98c2-f10cea058b4d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh"
Oct 31 21:45:00.308620 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.308598    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9973b2ec-4365-42d4-98c2-f10cea058b4d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh\" (UID: \"9973b2ec-4365-42d4-98c2-f10cea058b4d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh"
Oct 31 21:45:00.329965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.304420    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9hwvp\""
Oct 31 21:45:00.337772 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.337717    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/575790f7-e8f3-49bb-8d95-60f745bc8cff-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq\" (UID: \"575790f7-e8f3-49bb-8d95-60f745bc8cff\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq"
Oct 31 21:45:00.342177 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.342147    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq"]
Oct 31 21:45:00.351266 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.351239    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh"]
Oct 31 21:45:00.354324 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.344567    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq"
Oct 31 21:45:00.368994 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.367302    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shn5n\" (UniqueName: \"kubernetes.io/projected/575790f7-e8f3-49bb-8d95-60f745bc8cff-kube-api-access-shn5n\") pod \"maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq\" (UID: \"575790f7-e8f3-49bb-8d95-60f745bc8cff\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq"
Oct 31 21:45:00.373608 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.373580    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq"]
Oct 31 21:45:00.410496 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.410457    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28c4p\" (UniqueName: \"kubernetes.io/projected/9973b2ec-4365-42d4-98c2-f10cea058b4d-kube-api-access-28c4p\") pod \"maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh\" (UID: \"9973b2ec-4365-42d4-98c2-f10cea058b4d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh"
Oct 31 21:45:00.411373 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.411354    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c9f8306-0024-4639-8cf8-7e2b0a78e033-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq\" (UID: \"6c9f8306-0024-4639-8cf8-7e2b0a78e033\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq"
Oct 31 21:45:00.411540 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.411527    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p24q2\" (UniqueName: \"kubernetes.io/projected/6c9f8306-0024-4639-8cf8-7e2b0a78e033-kube-api-access-p24q2\") pod \"maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq\" (UID: \"6c9f8306-0024-4639-8cf8-7e2b0a78e033\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq"
Oct 31 21:45:00.411655 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.411644    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9973b2ec-4365-42d4-98c2-f10cea058b4d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh\" (UID: \"9973b2ec-4365-42d4-98c2-f10cea058b4d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh"
Oct 31 21:45:00.412574 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.412268    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9973b2ec-4365-42d4-98c2-f10cea058b4d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh\" (UID: \"9973b2ec-4365-42d4-98c2-f10cea058b4d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh"
Oct 31 21:45:00.464799 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.464771    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28c4p\" (UniqueName: \"kubernetes.io/projected/9973b2ec-4365-42d4-98c2-f10cea058b4d-kube-api-access-28c4p\") pod \"maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh\" (UID: \"9973b2ec-4365-42d4-98c2-f10cea058b4d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh"
Oct 31 21:45:00.500463 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.500436    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:45:00.514602 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.514562    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c9f8306-0024-4639-8cf8-7e2b0a78e033-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq\" (UID: \"6c9f8306-0024-4639-8cf8-7e2b0a78e033\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq"
Oct 31 21:45:00.514782 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.514662    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p24q2\" (UniqueName: \"kubernetes.io/projected/6c9f8306-0024-4639-8cf8-7e2b0a78e033-kube-api-access-p24q2\") pod \"maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq\" (UID: \"6c9f8306-0024-4639-8cf8-7e2b0a78e033\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq"
Oct 31 21:45:00.515786 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.515753    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c9f8306-0024-4639-8cf8-7e2b0a78e033-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq\" (UID: \"6c9f8306-0024-4639-8cf8-7e2b0a78e033\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq"
Oct 31 21:45:00.548016 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.547626    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p24q2\" (UniqueName: \"kubernetes.io/projected/6c9f8306-0024-4639-8cf8-7e2b0a78e033-kube-api-access-p24q2\") pod \"maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq\" (UID: \"6c9f8306-0024-4639-8cf8-7e2b0a78e033\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq"
Oct 31 21:45:00.617355 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.616261    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-workload-socket\") pod \"3788ae03-d501-4b1f-89e2-c47c533ead04\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") "
Oct 31 21:45:00.617355 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.616301    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-credential-socket\") pod \"3788ae03-d501-4b1f-89e2-c47c533ead04\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") "
Oct 31 21:45:00.617355 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.616334    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3788ae03-d501-4b1f-89e2-c47c533ead04-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3788ae03-d501-4b1f-89e2-c47c533ead04-istiod-ca-cert\") pod \"3788ae03-d501-4b1f-89e2-c47c533ead04\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") "
Oct 31 21:45:00.617355 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.616367    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3788ae03-d501-4b1f-89e2-c47c533ead04-kube-api-access-mrhg5\" (UniqueName: \"kubernetes.io/projected/3788ae03-d501-4b1f-89e2-c47c533ead04-kube-api-access-mrhg5\") pod \"3788ae03-d501-4b1f-89e2-c47c533ead04\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") "
Oct 31 21:45:00.617355 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.616389    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3788ae03-d501-4b1f-89e2-c47c533ead04-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3788ae03-d501-4b1f-89e2-c47c533ead04-istio-podinfo\") pod \"3788ae03-d501-4b1f-89e2-c47c533ead04\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") "
Oct 31 21:45:00.617355 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.616435    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3788ae03-d501-4b1f-89e2-c47c533ead04-istio-token\" (UniqueName: \"kubernetes.io/projected/3788ae03-d501-4b1f-89e2-c47c533ead04-istio-token\") pod \"3788ae03-d501-4b1f-89e2-c47c533ead04\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") "
Oct 31 21:45:00.617355 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.616468    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-istio-data\") pod \"3788ae03-d501-4b1f-89e2-c47c533ead04\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") "
Oct 31 21:45:00.617355 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.616556    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-workload-certs\") pod \"3788ae03-d501-4b1f-89e2-c47c533ead04\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") "
Oct 31 21:45:00.617355 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.616654    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-istio-envoy\") pod \"3788ae03-d501-4b1f-89e2-c47c533ead04\" (UID: \"3788ae03-d501-4b1f-89e2-c47c533ead04\") "
Oct 31 21:45:00.617355 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.616864    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-credential-socket" pod "3788ae03-d501-4b1f-89e2-c47c533ead04" (UID: "3788ae03-d501-4b1f-89e2-c47c533ead04"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:45:00.617355 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.616896    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-istio-data" pod "3788ae03-d501-4b1f-89e2-c47c533ead04" (UID: "3788ae03-d501-4b1f-89e2-c47c533ead04"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:45:00.617355 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.617041    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-workload-certs" pod "3788ae03-d501-4b1f-89e2-c47c533ead04" (UID: "3788ae03-d501-4b1f-89e2-c47c533ead04"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:45:00.617355 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.617200    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-workload-socket" pod "3788ae03-d501-4b1f-89e2-c47c533ead04" (UID: "3788ae03-d501-4b1f-89e2-c47c533ead04"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:45:00.617355 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.617250    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:00.617355 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.617265    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:00.617355 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.617282    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:00.617355 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.617316    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3788ae03-d501-4b1f-89e2-c47c533ead04-istiod-ca-cert" pod "3788ae03-d501-4b1f-89e2-c47c533ead04" (UID: "3788ae03-d501-4b1f-89e2-c47c533ead04"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:45:00.619072 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.619045    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3788ae03-d501-4b1f-89e2-c47c533ead04-istio-token" pod "3788ae03-d501-4b1f-89e2-c47c533ead04" (UID: "3788ae03-d501-4b1f-89e2-c47c533ead04"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:45:00.619072 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.619059    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3788ae03-d501-4b1f-89e2-c47c533ead04-kube-api-access-mrhg5" pod "3788ae03-d501-4b1f-89e2-c47c533ead04" (UID: "3788ae03-d501-4b1f-89e2-c47c533ead04"). InnerVolumeSpecName "kube-api-access-mrhg5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:45:00.619328 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.619305    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3788ae03-d501-4b1f-89e2-c47c533ead04-istio-podinfo" pod "3788ae03-d501-4b1f-89e2-c47c533ead04" (UID: "3788ae03-d501-4b1f-89e2-c47c533ead04"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:45:00.619393 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.619379    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-istio-envoy" pod "3788ae03-d501-4b1f-89e2-c47c533ead04" (UID: "3788ae03-d501-4b1f-89e2-c47c533ead04"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:45:00.666966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.666892    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq"
Oct 31 21:45:00.671215 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.671003    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh"
Oct 31 21:45:00.693011 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.692989    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq"
Oct 31 21:45:00.718777 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.718692    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3788ae03-d501-4b1f-89e2-c47c533ead04-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:00.718777 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.718716    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrhg5\" (UniqueName: \"kubernetes.io/projected/3788ae03-d501-4b1f-89e2-c47c533ead04-kube-api-access-mrhg5\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:00.718777 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.718729    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3788ae03-d501-4b1f-89e2-c47c533ead04-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:00.718777 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.718743    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3788ae03-d501-4b1f-89e2-c47c533ead04-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:00.718777 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.718752    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:00.718777 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.718759    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3788ae03-d501-4b1f-89e2-c47c533ead04-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:00.977266 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.977228    4460 generic.go:356] "Generic (PLEG): container finished" podID="3788ae03-d501-4b1f-89e2-c47c533ead04" containerID="2300bad6f458d416ac41c8241319d446675d7bae06d11947436c715898284d6c" exitCode=0
Oct 31 21:45:00.977467 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.977384    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9" event={"ID":"3788ae03-d501-4b1f-89e2-c47c533ead04","Type":"ContainerDied","Data":"2300bad6f458d416ac41c8241319d446675d7bae06d11947436c715898284d6c"}
Oct 31 21:45:00.977467 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.977444    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9" event={"ID":"3788ae03-d501-4b1f-89e2-c47c533ead04","Type":"ContainerDied","Data":"fa2d46065abd4ff70c664284e06494bcb6bbe2f37bde485a780209f429a75910"}
Oct 31 21:45:00.977603 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.977478    4460 scope.go:122] "RemoveContainer" containerID="2300bad6f458d416ac41c8241319d446675d7bae06d11947436c715898284d6c"
Oct 31 21:45:00.977668 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.977655    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"
Oct 31 21:45:00.996439 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:00.996412    4460 scope.go:122] "RemoveContainer" containerID="7dbb1b345324e0163f7ab4bc458a9797dae160c24aa274230203fa861d6b0197"
Oct 31 21:45:01.009370 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:01.009345    4460 scope.go:122] "RemoveContainer" containerID="2300bad6f458d416ac41c8241319d446675d7bae06d11947436c715898284d6c"
Oct 31 21:45:01.009709 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:45:01.009681    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2300bad6f458d416ac41c8241319d446675d7bae06d11947436c715898284d6c\": container with ID starting with 2300bad6f458d416ac41c8241319d446675d7bae06d11947436c715898284d6c not found: ID does not exist" containerID="2300bad6f458d416ac41c8241319d446675d7bae06d11947436c715898284d6c"
Oct 31 21:45:01.009786 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:01.009721    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2300bad6f458d416ac41c8241319d446675d7bae06d11947436c715898284d6c"} err="failed to get container status \"2300bad6f458d416ac41c8241319d446675d7bae06d11947436c715898284d6c\": rpc error: code = NotFound desc = could not find container \"2300bad6f458d416ac41c8241319d446675d7bae06d11947436c715898284d6c\": container with ID starting with 2300bad6f458d416ac41c8241319d446675d7bae06d11947436c715898284d6c not found: ID does not exist"
Oct 31 21:45:01.009786 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:01.009739    4460 scope.go:122] "RemoveContainer" containerID="7dbb1b345324e0163f7ab4bc458a9797dae160c24aa274230203fa861d6b0197"
Oct 31 21:45:01.010053 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:45:01.010034    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7dbb1b345324e0163f7ab4bc458a9797dae160c24aa274230203fa861d6b0197\": container with ID starting with 7dbb1b345324e0163f7ab4bc458a9797dae160c24aa274230203fa861d6b0197 not found: ID does not exist" containerID="7dbb1b345324e0163f7ab4bc458a9797dae160c24aa274230203fa861d6b0197"
Oct 31 21:45:01.010129 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:01.010060    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dbb1b345324e0163f7ab4bc458a9797dae160c24aa274230203fa861d6b0197"} err="failed to get container status \"7dbb1b345324e0163f7ab4bc458a9797dae160c24aa274230203fa861d6b0197\": rpc error: code = NotFound desc = could not find container \"7dbb1b345324e0163f7ab4bc458a9797dae160c24aa274230203fa861d6b0197\": container with ID starting with 7dbb1b345324e0163f7ab4bc458a9797dae160c24aa274230203fa861d6b0197 not found: ID does not exist"
Oct 31 21:45:01.074655 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:45:01.069766    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod575790f7_e8f3_49bb_8d95_60f745bc8cff.slice/crio-2a32533fbc300c1dbef3e65492593bf014c92f245907716d0e78019f6c6249e6 WatchSource:0}: Error finding container 2a32533fbc300c1dbef3e65492593bf014c92f245907716d0e78019f6c6249e6: Status 404 returned error can't find the container with id 2a32533fbc300c1dbef3e65492593bf014c92f245907716d0e78019f6c6249e6
Oct 31 21:45:01.076719 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:01.075915    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq"]
Oct 31 21:45:01.077403 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:45:01.077362    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c9f8306_0024_4639_8cf8_7e2b0a78e033.slice/crio-0dc5f3175a642b4b0ca69f7c8354c8c52c968e9adeea427cfe62a2c4870b4546 WatchSource:0}: Error finding container 0dc5f3175a642b4b0ca69f7c8354c8c52c968e9adeea427cfe62a2c4870b4546: Status 404 returned error can't find the container with id 0dc5f3175a642b4b0ca69f7c8354c8c52c968e9adeea427cfe62a2c4870b4546
Oct 31 21:45:01.078525 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:45:01.078496    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9973b2ec_4365_42d4_98c2_f10cea058b4d.slice/crio-63396c050f90a8437d9ef069e9dd90d490bfa8a49a45250f7a8fcebb8b9fd0b4 WatchSource:0}: Error finding container 63396c050f90a8437d9ef069e9dd90d490bfa8a49a45250f7a8fcebb8b9fd0b4: Status 404 returned error can't find the container with id 63396c050f90a8437d9ef069e9dd90d490bfa8a49a45250f7a8fcebb8b9fd0b4
Oct 31 21:45:01.086886 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:01.086862    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"]
Oct 31 21:45:01.092319 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:01.092297    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh"]
Oct 31 21:45:01.096270 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:01.096244    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-a-1d8f7b-openshift-default-7db6dd9597-rjkr9"]
Oct 31 21:45:01.099195 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:01.099177    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq"]
Oct 31 21:45:01.267623 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:01.267589    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:01.984998 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:01.984964    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh" event={"ID":"9973b2ec-4365-42d4-98c2-f10cea058b4d","Type":"ContainerStarted","Data":"63396c050f90a8437d9ef069e9dd90d490bfa8a49a45250f7a8fcebb8b9fd0b4"}
Oct 31 21:45:01.986366 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:01.986336    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq" event={"ID":"575790f7-e8f3-49bb-8d95-60f745bc8cff","Type":"ContainerStarted","Data":"2a32533fbc300c1dbef3e65492593bf014c92f245907716d0e78019f6c6249e6"}
Oct 31 21:45:01.987746 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:01.987718    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq" event={"ID":"6c9f8306-0024-4639-8cf8-7e2b0a78e033","Type":"ContainerStarted","Data":"0dc5f3175a642b4b0ca69f7c8354c8c52c968e9adeea427cfe62a2c4870b4546"}
Oct 31 21:45:02.258369 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:02.258286    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3788ae03-d501-4b1f-89e2-c47c533ead04" path="/var/lib/kubelet/pods/3788ae03-d501-4b1f-89e2-c47c533ead04/volumes"
Oct 31 21:45:03.054959 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:03.048821    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:04.748029 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:04.747989    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:06.027786 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:06.027748    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq" event={"ID":"6c9f8306-0024-4639-8cf8-7e2b0a78e033","Type":"ContainerStarted","Data":"73ec2c84644d766c627d9eedaac5f78e7501c7d34cc612343c87c3c464b9d18b"}
Oct 31 21:45:06.029566 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:06.029535    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh" event={"ID":"9973b2ec-4365-42d4-98c2-f10cea058b4d","Type":"ContainerStarted","Data":"69cded1d15d306e6d4000fa358db6a4f7fa10b2e65b145668f01fd24c3114fd0"}
Oct 31 21:45:06.031307 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:06.031285    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq" event={"ID":"575790f7-e8f3-49bb-8d95-60f745bc8cff","Type":"ContainerStarted","Data":"8a929f5b95ff9c5204864874dda17de6c8dd5e6ab4b36c5e4e02cc7f1a4c8f3a"}
Oct 31 21:45:06.071682 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:06.071642    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq" podStartSLOduration=1.8420612219999999 podStartE2EDuration="6.071632165s" totalImagesPullingTime="4.229570943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:45:06.071591047 +0000 UTC m=+1745.987900475" watchObservedRunningTime="2026-10-31 21:45:06.071632165 +0000 UTC m=+1745.987941593"
Oct 31 21:45:06.121757 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:06.121710    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq" podStartSLOduration=2.038182138 podStartE2EDuration="6.121694052s" totalImagesPullingTime="4.083511914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:45:06.119954007 +0000 UTC m=+1746.036263434" watchObservedRunningTime="2026-10-31 21:45:06.121694052 +0000 UTC m=+1746.038003480"
Oct 31 21:45:06.164176 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:06.164124    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh" podStartSLOduration=1.910366948 podStartE2EDuration="6.164110523s" totalImagesPullingTime="4.253743575s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:45:06.163116166 +0000 UTC m=+1746.079425596" watchObservedRunningTime="2026-10-31 21:45:06.164110523 +0000 UTC m=+1746.080419951"
Oct 31 21:45:06.204187 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:06.204154    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:06.417151 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:06.417098    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:06.533310 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:06.533278    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:07.043774 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:07.043741    4460 generic.go:356] "Generic (PLEG): container finished" podID="575790f7-e8f3-49bb-8d95-60f745bc8cff" containerID="8a929f5b95ff9c5204864874dda17de6c8dd5e6ab4b36c5e4e02cc7f1a4c8f3a" exitCode=60
Oct 31 21:45:07.044416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:07.043823    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq" event={"ID":"575790f7-e8f3-49bb-8d95-60f745bc8cff","Type":"ContainerDied","Data":"8a929f5b95ff9c5204864874dda17de6c8dd5e6ab4b36c5e4e02cc7f1a4c8f3a"}
Oct 31 21:45:07.044416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:07.044322    4460 scope.go:122] "RemoveContainer" containerID="8a929f5b95ff9c5204864874dda17de6c8dd5e6ab4b36c5e4e02cc7f1a4c8f3a"
Oct 31 21:45:07.045775 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:07.045747    4460 generic.go:356] "Generic (PLEG): container finished" podID="6c9f8306-0024-4639-8cf8-7e2b0a78e033" containerID="73ec2c84644d766c627d9eedaac5f78e7501c7d34cc612343c87c3c464b9d18b" exitCode=60
Oct 31 21:45:07.045912 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:07.045833    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq" event={"ID":"6c9f8306-0024-4639-8cf8-7e2b0a78e033","Type":"ContainerDied","Data":"73ec2c84644d766c627d9eedaac5f78e7501c7d34cc612343c87c3c464b9d18b"}
Oct 31 21:45:07.046158 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:07.046138    4460 scope.go:122] "RemoveContainer" containerID="73ec2c84644d766c627d9eedaac5f78e7501c7d34cc612343c87c3c464b9d18b"
Oct 31 21:45:07.048980 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:07.048954    4460 generic.go:356] "Generic (PLEG): container finished" podID="9973b2ec-4365-42d4-98c2-f10cea058b4d" containerID="69cded1d15d306e6d4000fa358db6a4f7fa10b2e65b145668f01fd24c3114fd0" exitCode=60
Oct 31 21:45:07.049114 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:07.049088    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh" event={"ID":"9973b2ec-4365-42d4-98c2-f10cea058b4d","Type":"ContainerDied","Data":"69cded1d15d306e6d4000fa358db6a4f7fa10b2e65b145668f01fd24c3114fd0"}
Oct 31 21:45:07.049360 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:07.049332    4460 scope.go:122] "RemoveContainer" containerID="69cded1d15d306e6d4000fa358db6a4f7fa10b2e65b145668f01fd24c3114fd0"
Oct 31 21:45:08.057814 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:08.057783    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh" event={"ID":"9973b2ec-4365-42d4-98c2-f10cea058b4d","Type":"ContainerStarted","Data":"8aeb081bca029b2a29aeb10991488ceade84c65afaa3a3565bf3dae1c019babf"}
Oct 31 21:45:08.059880 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:08.059856    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq" event={"ID":"575790f7-e8f3-49bb-8d95-60f745bc8cff","Type":"ContainerStarted","Data":"9afc181f50f8f41ca25b4c04c21246fa54276c20cddd1319281f25d3cb82b4ef"}
Oct 31 21:45:08.061859 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:08.061834    4460 generic.go:356] "Generic (PLEG): container finished" podID="6c9f8306-0024-4639-8cf8-7e2b0a78e033" containerID="f883130aba6ad381bd93ff1b4b39797c95d88f9995182c3a9130bd70ab4940c4" exitCode=60
Oct 31 21:45:08.061972 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:08.061883    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq" event={"ID":"6c9f8306-0024-4639-8cf8-7e2b0a78e033","Type":"ContainerDied","Data":"f883130aba6ad381bd93ff1b4b39797c95d88f9995182c3a9130bd70ab4940c4"}
Oct 31 21:45:08.061972 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:08.061918    4460 scope.go:122] "RemoveContainer" containerID="73ec2c84644d766c627d9eedaac5f78e7501c7d34cc612343c87c3c464b9d18b"
Oct 31 21:45:08.062187 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:08.062168    4460 scope.go:122] "RemoveContainer" containerID="f883130aba6ad381bd93ff1b4b39797c95d88f9995182c3a9130bd70ab4940c4"
Oct 31 21:45:08.062450 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:45:08.062428    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq_odh-ai-gateway-infra(6c9f8306-0024-4639-8cf8-7e2b0a78e033)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq" podUID="6c9f8306-0024-4639-8cf8-7e2b0a78e033"
Oct 31 21:45:08.731021 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:08.730988    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:08.936907 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:08.936875    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:09.070050 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:09.070018    4460 generic.go:356] "Generic (PLEG): container finished" podID="9973b2ec-4365-42d4-98c2-f10cea058b4d" containerID="8aeb081bca029b2a29aeb10991488ceade84c65afaa3a3565bf3dae1c019babf" exitCode=60
Oct 31 21:45:09.070512 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:09.070107    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh" event={"ID":"9973b2ec-4365-42d4-98c2-f10cea058b4d","Type":"ContainerDied","Data":"8aeb081bca029b2a29aeb10991488ceade84c65afaa3a3565bf3dae1c019babf"}
Oct 31 21:45:09.070512 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:09.070173    4460 scope.go:122] "RemoveContainer" containerID="69cded1d15d306e6d4000fa358db6a4f7fa10b2e65b145668f01fd24c3114fd0"
Oct 31 21:45:09.070512 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:09.070492    4460 scope.go:122] "RemoveContainer" containerID="8aeb081bca029b2a29aeb10991488ceade84c65afaa3a3565bf3dae1c019babf"
Oct 31 21:45:09.070899 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:45:09.070878    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh_odh-ai-gateway-infra(9973b2ec-4365-42d4-98c2-f10cea058b4d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh" podUID="9973b2ec-4365-42d4-98c2-f10cea058b4d"
Oct 31 21:45:09.072321 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:09.072299    4460 generic.go:356] "Generic (PLEG): container finished" podID="575790f7-e8f3-49bb-8d95-60f745bc8cff" containerID="9afc181f50f8f41ca25b4c04c21246fa54276c20cddd1319281f25d3cb82b4ef" exitCode=60
Oct 31 21:45:09.072426 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:09.072380    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq" event={"ID":"575790f7-e8f3-49bb-8d95-60f745bc8cff","Type":"ContainerDied","Data":"9afc181f50f8f41ca25b4c04c21246fa54276c20cddd1319281f25d3cb82b4ef"}
Oct 31 21:45:09.072624 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:09.072611    4460 scope.go:122] "RemoveContainer" containerID="9afc181f50f8f41ca25b4c04c21246fa54276c20cddd1319281f25d3cb82b4ef"
Oct 31 21:45:09.072955 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:45:09.072914    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq_odh-ai-gateway-infra(575790f7-e8f3-49bb-8d95-60f745bc8cff)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq" podUID="575790f7-e8f3-49bb-8d95-60f745bc8cff"
Oct 31 21:45:09.074433 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:09.074417    4460 scope.go:122] "RemoveContainer" containerID="f883130aba6ad381bd93ff1b4b39797c95d88f9995182c3a9130bd70ab4940c4"
Oct 31 21:45:09.074708 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:45:09.074686    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq_odh-ai-gateway-infra(6c9f8306-0024-4639-8cf8-7e2b0a78e033)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq" podUID="6c9f8306-0024-4639-8cf8-7e2b0a78e033"
Oct 31 21:45:09.085493 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:09.085473    4460 scope.go:122] "RemoveContainer" containerID="8a929f5b95ff9c5204864874dda17de6c8dd5e6ab4b36c5e4e02cc7f1a4c8f3a"
Oct 31 21:45:10.082848 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:10.082811    4460 scope.go:122] "RemoveContainer" containerID="8aeb081bca029b2a29aeb10991488ceade84c65afaa3a3565bf3dae1c019babf"
Oct 31 21:45:10.083382 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:45:10.083236    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh_odh-ai-gateway-infra(9973b2ec-4365-42d4-98c2-f10cea058b4d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh" podUID="9973b2ec-4365-42d4-98c2-f10cea058b4d"
Oct 31 21:45:10.084168 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:10.084145    4460 generic.go:356] "Generic (PLEG): container finished" podID="78bb7115-5485-45ec-9b35-a76a75ba3eff" containerID="9b78df3386149b2efadc188f14b892e0d594f5cd5141821fc388f25309f65331" exitCode=0
Oct 31 21:45:10.084283 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:10.084226    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns" event={"ID":"78bb7115-5485-45ec-9b35-a76a75ba3eff","Type":"ContainerDied","Data":"9b78df3386149b2efadc188f14b892e0d594f5cd5141821fc388f25309f65331"}
Oct 31 21:45:10.084695 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:10.084671    4460 scope.go:122] "RemoveContainer" containerID="9b78df3386149b2efadc188f14b892e0d594f5cd5141821fc388f25309f65331"
Oct 31 21:45:10.086107 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:10.086093    4460 scope.go:122] "RemoveContainer" containerID="9afc181f50f8f41ca25b4c04c21246fa54276c20cddd1319281f25d3cb82b4ef"
Oct 31 21:45:10.086210 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:10.086188    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:45:10.086266 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:10.086253    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:45:10.086318 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:10.086282    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:45:10.086318 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:45:10.086293    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq_odh-ai-gateway-infra(575790f7-e8f3-49bb-8d95-60f745bc8cff)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq" podUID="575790f7-e8f3-49bb-8d95-60f745bc8cff"
Oct 31 21:45:10.157416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:10.157383    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:10.439578 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:10.439485    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:10.641414 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:10.640681    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:10.997759 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:10.995692    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:11.094752 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:11.094718    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns" event={"ID":"78bb7115-5485-45ec-9b35-a76a75ba3eff","Type":"ContainerStarted","Data":"41d8d0da12535ca5380761cf68aa470090d3a575d8ae88a7af62410240b8ddd9"}
Oct 31 21:45:11.462217 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:11.462177    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:45:12.001316 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:12.001278    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:45:12.002774 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:12.002748    4460 patch_prober.go:29] interesting pod/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused" start-of-body=
Oct 31 21:45:12.002891 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:12.002797    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns" podUID="78bb7115-5485-45ec-9b35-a76a75ba3eff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused"
Oct 31 21:45:12.162481 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:12.162442    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:13.001465 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:13.001386    4460 patch_prober.go:29] interesting pod/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused" start-of-body=
Oct 31 21:45:13.001465 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:13.001442    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns" podUID="78bb7115-5485-45ec-9b35-a76a75ba3eff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused"
Oct 31 21:45:13.952140 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:13.951861    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:14.001731 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:14.001694    4460 patch_prober.go:29] interesting pod/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused" start-of-body=
Oct 31 21:45:14.001865 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:14.001754    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns" podUID="78bb7115-5485-45ec-9b35-a76a75ba3eff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused"
Oct 31 21:45:14.065296 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:14.065263    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:15.001599 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:15.001564    4460 patch_prober.go:29] interesting pod/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused" start-of-body=
Oct 31 21:45:15.002047 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:15.001628    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns" podUID="78bb7115-5485-45ec-9b35-a76a75ba3eff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused"
Oct 31 21:45:15.693471 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:15.693425    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:15.806861 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:15.806826    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:15.918244 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:15.918207    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:16.046964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:16.040061    4460 patch_prober.go:29] interesting pod/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused" start-of-body=
Oct 31 21:45:16.046964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:16.040145    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns" podUID="78bb7115-5485-45ec-9b35-a76a75ba3eff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.129:15021/healthz/ready\": dial tcp 10.133.0.129:15021: connect: connection refused"
Oct 31 21:45:17.006043 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:17.006004    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:45:17.007234 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:17.007213    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:45:17.086479 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:17.086452    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-hxh82_bb3b6872-a3ff-438d-b1ba-45fca4844b6e/payload-processing/0.log"
Oct 31 21:45:17.312402 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:17.312374    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-hxh82_bb3b6872-a3ff-438d-b1ba-45fca4844b6e/payload-processing/0.log"
Oct 31 21:45:17.603648 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:17.603563    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:19.217507 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:19.217477    4460 scope.go:122] "RemoveContainer" containerID="8aeb081bca029b2a29aeb10991488ceade84c65afaa3a3565bf3dae1c019babf"
Oct 31 21:45:19.451968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:19.451925    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:20.218846 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:20.218813    4460 generic.go:356] "Generic (PLEG): container finished" podID="9973b2ec-4365-42d4-98c2-f10cea058b4d" containerID="a7febf48250af15912993202f248233bd4ea945a5f8dc7a1adb16382042a7153" exitCode=60
Oct 31 21:45:20.223599 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:20.223545    4460 scope.go:122] "RemoveContainer" containerID="9afc181f50f8f41ca25b4c04c21246fa54276c20cddd1319281f25d3cb82b4ef"
Oct 31 21:45:20.239489 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:20.239458    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh" event={"ID":"9973b2ec-4365-42d4-98c2-f10cea058b4d","Type":"ContainerDied","Data":"a7febf48250af15912993202f248233bd4ea945a5f8dc7a1adb16382042a7153"}
Oct 31 21:45:20.239618 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:20.239534    4460 scope.go:122] "RemoveContainer" containerID="8aeb081bca029b2a29aeb10991488ceade84c65afaa3a3565bf3dae1c019babf"
Oct 31 21:45:20.240067 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:20.240048    4460 scope.go:122] "RemoveContainer" containerID="a7febf48250af15912993202f248233bd4ea945a5f8dc7a1adb16382042a7153"
Oct 31 21:45:20.241240 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:45:20.241094    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh_odh-ai-gateway-infra(9973b2ec-4365-42d4-98c2-f10cea058b4d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh" podUID="9973b2ec-4365-42d4-98c2-f10cea058b4d"
Oct 31 21:45:21.218408 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:21.218379    4460 scope.go:122] "RemoveContainer" containerID="f883130aba6ad381bd93ff1b4b39797c95d88f9995182c3a9130bd70ab4940c4"
Oct 31 21:45:21.228729 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:21.228705    4460 generic.go:356] "Generic (PLEG): container finished" podID="575790f7-e8f3-49bb-8d95-60f745bc8cff" containerID="28cd3df72d35e42a6ec48515ee47edc6191a691646d791a00b4fb300b68f3764" exitCode=60
Oct 31 21:45:21.229127 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:21.228776    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq" event={"ID":"575790f7-e8f3-49bb-8d95-60f745bc8cff","Type":"ContainerDied","Data":"28cd3df72d35e42a6ec48515ee47edc6191a691646d791a00b4fb300b68f3764"}
Oct 31 21:45:21.229127 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:21.228822    4460 scope.go:122] "RemoveContainer" containerID="9afc181f50f8f41ca25b4c04c21246fa54276c20cddd1319281f25d3cb82b4ef"
Oct 31 21:45:21.229216 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:21.229170    4460 scope.go:122] "RemoveContainer" containerID="28cd3df72d35e42a6ec48515ee47edc6191a691646d791a00b4fb300b68f3764"
Oct 31 21:45:21.229417 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:45:21.229398    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq_odh-ai-gateway-infra(575790f7-e8f3-49bb-8d95-60f745bc8cff)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq" podUID="575790f7-e8f3-49bb-8d95-60f745bc8cff"
Oct 31 21:45:21.260806 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:21.260775    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:21.311623 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:21.311463    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh"]
Oct 31 21:45:21.457570 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:45:21.457529    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:45:21.570402 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:21.570351    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh"
Oct 31 21:45:21.677006 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:21.676972    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9973b2ec-4365-42d4-98c2-f10cea058b4d-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9973b2ec-4365-42d4-98c2-f10cea058b4d-maas-api-service-ca\") pod \"9973b2ec-4365-42d4-98c2-f10cea058b4d\" (UID: \"9973b2ec-4365-42d4-98c2-f10cea058b4d\") "
Oct 31 21:45:21.677166 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:21.677023    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9973b2ec-4365-42d4-98c2-f10cea058b4d-kube-api-access-28c4p\" (UniqueName: \"kubernetes.io/projected/9973b2ec-4365-42d4-98c2-f10cea058b4d-kube-api-access-28c4p\") pod \"9973b2ec-4365-42d4-98c2-f10cea058b4d\" (UID: \"9973b2ec-4365-42d4-98c2-f10cea058b4d\") "
Oct 31 21:45:21.677367 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:21.677343    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9973b2ec-4365-42d4-98c2-f10cea058b4d-maas-api-service-ca" pod "9973b2ec-4365-42d4-98c2-f10cea058b4d" (UID: "9973b2ec-4365-42d4-98c2-f10cea058b4d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:45:21.677601 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:21.677582    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9973b2ec-4365-42d4-98c2-f10cea058b4d-maas-api-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:21.679116 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:21.679095    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9973b2ec-4365-42d4-98c2-f10cea058b4d-kube-api-access-28c4p" pod "9973b2ec-4365-42d4-98c2-f10cea058b4d" (UID: "9973b2ec-4365-42d4-98c2-f10cea058b4d"). InnerVolumeSpecName "kube-api-access-28c4p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:45:21.778804 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:21.778774    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28c4p\" (UniqueName: \"kubernetes.io/projected/9973b2ec-4365-42d4-98c2-f10cea058b4d-kube-api-access-28c4p\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:22.236751 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:22.236724    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh"
Oct 31 21:45:22.236751 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:22.236739    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh" event={"ID":"9973b2ec-4365-42d4-98c2-f10cea058b4d","Type":"ContainerDied","Data":"63396c050f90a8437d9ef069e9dd90d490bfa8a49a45250f7a8fcebb8b9fd0b4"}
Oct 31 21:45:22.237223 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:22.236800    4460 scope.go:122] "RemoveContainer" containerID="a7febf48250af15912993202f248233bd4ea945a5f8dc7a1adb16382042a7153"
Oct 31 21:45:22.241068 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:22.241046    4460 generic.go:356] "Generic (PLEG): container finished" podID="6c9f8306-0024-4639-8cf8-7e2b0a78e033" containerID="6150736421f1458dc90b1fab8bd4089769a09d8858b20cdc854c47c7a17a2e7e" exitCode=60
Oct 31 21:45:22.241183 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:22.241087    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq" event={"ID":"6c9f8306-0024-4639-8cf8-7e2b0a78e033","Type":"ContainerDied","Data":"6150736421f1458dc90b1fab8bd4089769a09d8858b20cdc854c47c7a17a2e7e"}
Oct 31 21:45:22.241468 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:22.241452    4460 scope.go:122] "RemoveContainer" containerID="6150736421f1458dc90b1fab8bd4089769a09d8858b20cdc854c47c7a17a2e7e"
Oct 31 21:45:22.241711 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:45:22.241691    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq_odh-ai-gateway-infra(6c9f8306-0024-4639-8cf8-7e2b0a78e033)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq" podUID="6c9f8306-0024-4639-8cf8-7e2b0a78e033"
Oct 31 21:45:22.247126 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:22.247111    4460 scope.go:122] "RemoveContainer" containerID="f883130aba6ad381bd93ff1b4b39797c95d88f9995182c3a9130bd70ab4940c4"
Oct 31 21:45:22.287126 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:22.287094    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq"]
Oct 31 21:45:22.338557 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:22.338510    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh"]
Oct 31 21:45:22.341976 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:22.341934    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4b5989-29891385-fcbqh"]
Oct 31 21:45:22.455360 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:22.455337    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq"
Oct 31 21:45:22.495036 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:22.495001    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/575790f7-e8f3-49bb-8d95-60f745bc8cff-kube-api-access-shn5n\" (UniqueName: \"kubernetes.io/projected/575790f7-e8f3-49bb-8d95-60f745bc8cff-kube-api-access-shn5n\") pod \"575790f7-e8f3-49bb-8d95-60f745bc8cff\" (UID: \"575790f7-e8f3-49bb-8d95-60f745bc8cff\") "
Oct 31 21:45:22.495199 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:22.495151    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/575790f7-e8f3-49bb-8d95-60f745bc8cff-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/575790f7-e8f3-49bb-8d95-60f745bc8cff-maas-api-service-ca\") pod \"575790f7-e8f3-49bb-8d95-60f745bc8cff\" (UID: \"575790f7-e8f3-49bb-8d95-60f745bc8cff\") "
Oct 31 21:45:22.495624 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:22.495594    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/575790f7-e8f3-49bb-8d95-60f745bc8cff-maas-api-service-ca" pod "575790f7-e8f3-49bb-8d95-60f745bc8cff" (UID: "575790f7-e8f3-49bb-8d95-60f745bc8cff"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:45:22.495768 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:22.495742    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/575790f7-e8f3-49bb-8d95-60f745bc8cff-maas-api-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:22.497512 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:22.497485    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/575790f7-e8f3-49bb-8d95-60f745bc8cff-kube-api-access-shn5n" pod "575790f7-e8f3-49bb-8d95-60f745bc8cff" (UID: "575790f7-e8f3-49bb-8d95-60f745bc8cff"). InnerVolumeSpecName "kube-api-access-shn5n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:45:22.597610 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:22.597575    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shn5n\" (UniqueName: \"kubernetes.io/projected/575790f7-e8f3-49bb-8d95-60f745bc8cff-kube-api-access-shn5n\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:23.071718 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:23.071682    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:23.248734 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:23.248703    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq"
Oct 31 21:45:23.249229 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:23.248735    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq" event={"ID":"575790f7-e8f3-49bb-8d95-60f745bc8cff","Type":"ContainerDied","Data":"2a32533fbc300c1dbef3e65492593bf014c92f245907716d0e78019f6c6249e6"}
Oct 31 21:45:23.249229 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:23.248792    4460 scope.go:122] "RemoveContainer" containerID="28cd3df72d35e42a6ec48515ee47edc6191a691646d791a00b4fb300b68f3764"
Oct 31 21:45:23.277913 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:23.277887    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:45:23.307755 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:23.307726    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq"]
Oct 31 21:45:23.315955 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:23.311471    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq"]
Oct 31 21:45:23.326553 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:23.326481    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-106555-29891385-tpqkq"]
Oct 31 21:45:23.481765 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:23.481742    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq"
Oct 31 21:45:23.611610 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:23.611526    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6c9f8306-0024-4639-8cf8-7e2b0a78e033-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c9f8306-0024-4639-8cf8-7e2b0a78e033-maas-api-service-ca\") pod \"6c9f8306-0024-4639-8cf8-7e2b0a78e033\" (UID: \"6c9f8306-0024-4639-8cf8-7e2b0a78e033\") "
Oct 31 21:45:23.611752 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:23.611706    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6c9f8306-0024-4639-8cf8-7e2b0a78e033-kube-api-access-p24q2\" (UniqueName: \"kubernetes.io/projected/6c9f8306-0024-4639-8cf8-7e2b0a78e033-kube-api-access-p24q2\") pod \"6c9f8306-0024-4639-8cf8-7e2b0a78e033\" (UID: \"6c9f8306-0024-4639-8cf8-7e2b0a78e033\") "
Oct 31 21:45:23.611884 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:23.611861    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c9f8306-0024-4639-8cf8-7e2b0a78e033-maas-api-service-ca" pod "6c9f8306-0024-4639-8cf8-7e2b0a78e033" (UID: "6c9f8306-0024-4639-8cf8-7e2b0a78e033"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:45:23.612239 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:23.612216    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c9f8306-0024-4639-8cf8-7e2b0a78e033-maas-api-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:23.613693 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:23.613669    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c9f8306-0024-4639-8cf8-7e2b0a78e033-kube-api-access-p24q2" pod "6c9f8306-0024-4639-8cf8-7e2b0a78e033" (UID: "6c9f8306-0024-4639-8cf8-7e2b0a78e033"). InnerVolumeSpecName "kube-api-access-p24q2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:45:23.713346 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:23.713318    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p24q2\" (UniqueName: \"kubernetes.io/projected/6c9f8306-0024-4639-8cf8-7e2b0a78e033-kube-api-access-p24q2\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:24.226410 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:24.226376    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="575790f7-e8f3-49bb-8d95-60f745bc8cff" path="/var/lib/kubelet/pods/575790f7-e8f3-49bb-8d95-60f745bc8cff/volumes"
Oct 31 21:45:24.226801 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:24.226784    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9973b2ec-4365-42d4-98c2-f10cea058b4d" path="/var/lib/kubelet/pods/9973b2ec-4365-42d4-98c2-f10cea058b4d/volumes"
Oct 31 21:45:24.260966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:24.260908    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq"
Oct 31 21:45:24.261401 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:24.260906    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq" event={"ID":"6c9f8306-0024-4639-8cf8-7e2b0a78e033","Type":"ContainerDied","Data":"0dc5f3175a642b4b0ca69f7c8354c8c52c968e9adeea427cfe62a2c4870b4546"}
Oct 31 21:45:24.261401 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:24.261038    4460 scope.go:122] "RemoveContainer" containerID="6150736421f1458dc90b1fab8bd4089769a09d8858b20cdc854c47c7a17a2e7e"
Oct 31 21:45:24.394194 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:24.386998    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq"]
Oct 31 21:45:24.417963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:24.414112    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-448b01-29891385-tlpnq"]
Oct 31 21:45:25.043303 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:25.042512    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm"]
Oct 31 21:45:25.043303 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:25.042767    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm" podUID="db04d59d-9b82-492d-b5dd-58d51b35d555" containerName="maas-api" containerID="cri-o://39715ef36f593cc626474aeb7efa425d7386fb591d39348090cfcfee034ed05a" gracePeriod=30
Oct 31 21:45:25.106175 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:25.106148    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:25.272749 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:25.270547    4460 generic.go:356] "Generic (PLEG): container finished" podID="db04d59d-9b82-492d-b5dd-58d51b35d555" containerID="39715ef36f593cc626474aeb7efa425d7386fb591d39348090cfcfee034ed05a" exitCode=0
Oct 31 21:45:25.272749 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:25.270664    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm" event={"ID":"db04d59d-9b82-492d-b5dd-58d51b35d555","Type":"ContainerDied","Data":"39715ef36f593cc626474aeb7efa425d7386fb591d39348090cfcfee034ed05a"}
Oct 31 21:45:25.583382 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:25.583351    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm"
Oct 31 21:45:25.644960 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:25.644328    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db04d59d-9b82-492d-b5dd-58d51b35d555-openshift-service-ca" pod "db04d59d-9b82-492d-b5dd-58d51b35d555" (UID: "db04d59d-9b82-492d-b5dd-58d51b35d555"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:45:25.644960 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:25.644388    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/db04d59d-9b82-492d-b5dd-58d51b35d555-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db04d59d-9b82-492d-b5dd-58d51b35d555-openshift-service-ca\") pod \"db04d59d-9b82-492d-b5dd-58d51b35d555\" (UID: \"db04d59d-9b82-492d-b5dd-58d51b35d555\") "
Oct 31 21:45:25.644960 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:25.644460    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db04d59d-9b82-492d-b5dd-58d51b35d555-kube-api-access-xghgr\" (UniqueName: \"kubernetes.io/projected/db04d59d-9b82-492d-b5dd-58d51b35d555-kube-api-access-xghgr\") pod \"db04d59d-9b82-492d-b5dd-58d51b35d555\" (UID: \"db04d59d-9b82-492d-b5dd-58d51b35d555\") "
Oct 31 21:45:25.648957 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:25.645820    4460 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db04d59d-9b82-492d-b5dd-58d51b35d555-openshift-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:25.677451 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:25.668901    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db04d59d-9b82-492d-b5dd-58d51b35d555-kube-api-access-xghgr" pod "db04d59d-9b82-492d-b5dd-58d51b35d555" (UID: "db04d59d-9b82-492d-b5dd-58d51b35d555"). InnerVolumeSpecName "kube-api-access-xghgr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:45:25.677451 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:25.669906    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:25.746415 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:25.746344    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/db04d59d-9b82-492d-b5dd-58d51b35d555-metrics-certs\" (UniqueName: \"kubernetes.io/secret/db04d59d-9b82-492d-b5dd-58d51b35d555-metrics-certs\") pod \"db04d59d-9b82-492d-b5dd-58d51b35d555\" (UID: \"db04d59d-9b82-492d-b5dd-58d51b35d555\") "
Oct 31 21:45:25.746568 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:25.746460    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/db04d59d-9b82-492d-b5dd-58d51b35d555-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db04d59d-9b82-492d-b5dd-58d51b35d555-maas-api-tls\") pod \"db04d59d-9b82-492d-b5dd-58d51b35d555\" (UID: \"db04d59d-9b82-492d-b5dd-58d51b35d555\") "
Oct 31 21:45:25.747229 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:25.747201    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xghgr\" (UniqueName: \"kubernetes.io/projected/db04d59d-9b82-492d-b5dd-58d51b35d555-kube-api-access-xghgr\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:25.748554 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:25.748527    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db04d59d-9b82-492d-b5dd-58d51b35d555-maas-api-tls" pod "db04d59d-9b82-492d-b5dd-58d51b35d555" (UID: "db04d59d-9b82-492d-b5dd-58d51b35d555"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 21:45:25.748656 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:25.748572    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db04d59d-9b82-492d-b5dd-58d51b35d555-metrics-certs" pod "db04d59d-9b82-492d-b5dd-58d51b35d555" (UID: "db04d59d-9b82-492d-b5dd-58d51b35d555"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 21:45:25.790151 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:25.790111    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:25.848918 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:25.848887    4460 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db04d59d-9b82-492d-b5dd-58d51b35d555-metrics-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:25.848918 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:25.848915    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db04d59d-9b82-492d-b5dd-58d51b35d555-maas-api-tls\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:25.915014 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:25.912438    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:26.225722 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:26.225686    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6c9f8306-0024-4639-8cf8-7e2b0a78e033" path="/var/lib/kubelet/pods/6c9f8306-0024-4639-8cf8-7e2b0a78e033/volumes"
Oct 31 21:45:26.280502 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:26.280465    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm"
Oct 31 21:45:26.280927 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:26.280506    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm" event={"ID":"db04d59d-9b82-492d-b5dd-58d51b35d555","Type":"ContainerDied","Data":"4c775dea4f5418173d08dbc2aa88f733341b6b6cb3b8ca6c204eb337678f4e44"}
Oct 31 21:45:26.280927 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:26.280564    4460 scope.go:122] "RemoveContainer" containerID="39715ef36f593cc626474aeb7efa425d7386fb591d39348090cfcfee034ed05a"
Oct 31 21:45:26.308626 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:26.308599    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:26.340761 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:26.340736    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm"]
Oct 31 21:45:26.354046 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:26.354019    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-f4569e-75dc768bf7-829tm"]
Oct 31 21:45:27.754022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:27.753983    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:28.225684 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:28.225646    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="db04d59d-9b82-492d-b5dd-58d51b35d555" path="/var/lib/kubelet/pods/db04d59d-9b82-492d-b5dd-58d51b35d555/volumes"
Oct 31 21:45:29.495793 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:29.495759    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:30.935975 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:30.935925    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:31.128975 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.128910    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:31.239286 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.239213    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:31.526700 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.526640    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:31.564822 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.564787    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"]
Oct 31 21:45:31.566107 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.566080    4460 cpu_manager.go:414] "containerMap: removing container" podUID="6c9f8306-0024-4639-8cf8-7e2b0a78e033" containerName="cleanup"
Oct 31 21:45:31.574652 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.574623    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6c9f8306-0024-4639-8cf8-7e2b0a78e033" containerName="cleanup"
Oct 31 21:45:31.574872 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.574856    4460 cpu_manager.go:414] "containerMap: removing container" podUID="6c9f8306-0024-4639-8cf8-7e2b0a78e033" containerName="cleanup"
Oct 31 21:45:31.575051 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.575034    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6c9f8306-0024-4639-8cf8-7e2b0a78e033" containerName="cleanup"
Oct 31 21:45:31.575588 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.575569    4460 cpu_manager.go:414] "containerMap: removing container" podUID="9973b2ec-4365-42d4-98c2-f10cea058b4d" containerName="cleanup"
Oct 31 21:45:31.575723 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.575710    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9973b2ec-4365-42d4-98c2-f10cea058b4d" containerName="cleanup"
Oct 31 21:45:31.575824 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.575815    4460 cpu_manager.go:414] "containerMap: removing container" podUID="9973b2ec-4365-42d4-98c2-f10cea058b4d" containerName="cleanup"
Oct 31 21:45:31.575899 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.575887    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9973b2ec-4365-42d4-98c2-f10cea058b4d" containerName="cleanup"
Oct 31 21:45:31.576017 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.576007    4460 cpu_manager.go:414] "containerMap: removing container" podUID="575790f7-e8f3-49bb-8d95-60f745bc8cff" containerName="cleanup"
Oct 31 21:45:31.576101 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.576091    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="575790f7-e8f3-49bb-8d95-60f745bc8cff" containerName="cleanup"
Oct 31 21:45:31.576192 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.576181    4460 cpu_manager.go:414] "containerMap: removing container" podUID="3788ae03-d501-4b1f-89e2-c47c533ead04" containerName="istio-proxy"
Oct 31 21:45:31.576364 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.576349    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3788ae03-d501-4b1f-89e2-c47c533ead04" containerName="istio-proxy"
Oct 31 21:45:31.576473 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.576461    4460 cpu_manager.go:414] "containerMap: removing container" podUID="db04d59d-9b82-492d-b5dd-58d51b35d555" containerName="maas-api"
Oct 31 21:45:31.576557 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.576546    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="db04d59d-9b82-492d-b5dd-58d51b35d555" containerName="maas-api"
Oct 31 21:45:31.576650 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.576640    4460 cpu_manager.go:414] "containerMap: removing container" podUID="9973b2ec-4365-42d4-98c2-f10cea058b4d" containerName="cleanup"
Oct 31 21:45:31.576748 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.576731    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9973b2ec-4365-42d4-98c2-f10cea058b4d" containerName="cleanup"
Oct 31 21:45:31.576809 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.576760    4460 cpu_manager.go:414] "containerMap: removing container" podUID="3788ae03-d501-4b1f-89e2-c47c533ead04" containerName="istio-proxy"
Oct 31 21:45:31.576809 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.576767    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3788ae03-d501-4b1f-89e2-c47c533ead04" containerName="istio-proxy"
Oct 31 21:45:31.576809 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.576783    4460 cpu_manager.go:414] "containerMap: removing container" podUID="575790f7-e8f3-49bb-8d95-60f745bc8cff" containerName="cleanup"
Oct 31 21:45:31.576809 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.576805    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="575790f7-e8f3-49bb-8d95-60f745bc8cff" containerName="cleanup"
Oct 31 21:45:31.577034 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.576816    4460 cpu_manager.go:414] "containerMap: removing container" podUID="575790f7-e8f3-49bb-8d95-60f745bc8cff" containerName="cleanup"
Oct 31 21:45:31.577034 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.576822    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="575790f7-e8f3-49bb-8d95-60f745bc8cff" containerName="cleanup"
Oct 31 21:45:31.577139 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.577096    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="6c9f8306-0024-4639-8cf8-7e2b0a78e033" containerName="cleanup"
Oct 31 21:45:31.577139 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.577130    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="6c9f8306-0024-4639-8cf8-7e2b0a78e033" containerName="cleanup"
Oct 31 21:45:31.577246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.577147    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="9973b2ec-4365-42d4-98c2-f10cea058b4d" containerName="cleanup"
Oct 31 21:45:31.577246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.577160    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="575790f7-e8f3-49bb-8d95-60f745bc8cff" containerName="cleanup"
Oct 31 21:45:31.577246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.577174    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="3788ae03-d501-4b1f-89e2-c47c533ead04" containerName="istio-proxy"
Oct 31 21:45:31.577246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.577184    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="db04d59d-9b82-492d-b5dd-58d51b35d555" containerName="maas-api"
Oct 31 21:45:31.577246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.577208    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="9973b2ec-4365-42d4-98c2-f10cea058b4d" containerName="cleanup"
Oct 31 21:45:31.577246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.577217    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="3788ae03-d501-4b1f-89e2-c47c533ead04" containerName="istio-proxy"
Oct 31 21:45:31.577246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.577227    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="575790f7-e8f3-49bb-8d95-60f745bc8cff" containerName="cleanup"
Oct 31 21:45:31.590705 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.589746    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:45:31.625205 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.625177    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"]
Oct 31 21:45:31.649422 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.649376    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-credential-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:45:31.649685 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.649664    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istio-envoy\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:45:31.649857 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.649844    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-workload-certs\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:45:31.649988 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.649974    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istio-podinfo\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:45:31.650124 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.650111    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-workload-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:45:31.650251 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.650237    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4b5qv\" (UniqueName: \"kubernetes.io/projected/3996ffdd-3dbe-41f6-ab61-62102a86e76b-kube-api-access-4b5qv\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:45:31.650372 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.650361    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istiod-ca-cert\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:45:31.650538 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.650525    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istio-token\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:45:31.650660 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.650647    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istio-data\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:45:31.695707 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.695667    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:31.751847 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.751818    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-workload-certs\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:45:31.752020 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.751858    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istio-podinfo\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:45:31.752020 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.751899    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-workload-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:45:31.752020 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.751972    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b5qv\" (UniqueName: \"kubernetes.io/projected/3996ffdd-3dbe-41f6-ab61-62102a86e76b-kube-api-access-4b5qv\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:45:31.752178 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.752034    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istiod-ca-cert\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:45:31.752178 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.752132    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istio-token\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:45:31.752277 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.752179    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istio-data\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:45:31.752277 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.752257    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-workload-certs\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:45:31.752389 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.752262    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-credential-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:45:31.752389 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.752337    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istio-envoy\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:45:31.752538 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.752512    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istio-data\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:45:31.752609 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.752548    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-credential-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:45:31.752609 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.752576    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-workload-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:45:31.752926 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.752898    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istiod-ca-cert\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:45:31.754410 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.754390    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istio-podinfo\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:45:31.754544 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.754524    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istio-envoy\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:45:31.777137 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.777076    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4b5qv\" (UniqueName: \"kubernetes.io/projected/3996ffdd-3dbe-41f6-ab61-62102a86e76b-kube-api-access-4b5qv\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:45:31.777293 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.777273    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istio-token\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:45:31.811228 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.811176    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:31.927961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:31.921172    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:45:32.195657 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:32.195622    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:32.335359 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:32.335332    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"]
Oct 31 21:45:32.335491 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:45:32.335467    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3996ffdd_3dbe_41f6_ab61_62102a86e76b.slice/crio-fe55643b78f726637105bf3a9f1b640504e049150d24af079c91d7a6f5c28bd8 WatchSource:0}: Error finding container fe55643b78f726637105bf3a9f1b640504e049150d24af079c91d7a6f5c28bd8: Status 404 returned error can't find the container with id fe55643b78f726637105bf3a9f1b640504e049150d24af079c91d7a6f5c28bd8
Oct 31 21:45:32.337875 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:32.337843    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:45:32.337985 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:32.337916    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:45:32.337985 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:32.337977    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:45:33.084298 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:33.084262    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:33.207027 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:33.206990    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:33.351807 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:33.351702    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm" event={"ID":"3996ffdd-3dbe-41f6-ab61-62102a86e76b","Type":"ContainerStarted","Data":"860bf94ed996535b0710e5576dc042d5c8eb988d8fcd7ad646f2c39412a662d6"}
Oct 31 21:45:33.351807 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:33.351788    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm" event={"ID":"3996ffdd-3dbe-41f6-ab61-62102a86e76b","Type":"ContainerStarted","Data":"fe55643b78f726637105bf3a9f1b640504e049150d24af079c91d7a6f5c28bd8"}
Oct 31 21:45:33.405821 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:33.405785    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"]
Oct 31 21:45:33.406077 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:33.406046    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc" podUID="19fcad1c-5c6e-4274-936d-bf93ca76c7d4" containerName="istio-proxy" containerID="cri-o://9b1af9a77e34f42b623c8e37da558b7140d452b72431ff1c949b7ceb52ec6fff" gracePeriod=30
Oct 31 21:45:33.417758 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:33.417713    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm" podStartSLOduration=2.4177007919999998 podStartE2EDuration="2.417700792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:45:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:45:33.416136148 +0000 UTC m=+1773.332445578" watchObservedRunningTime="2026-10-31 21:45:33.417700792 +0000 UTC m=+1773.334010221"
Oct 31 21:45:33.447965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:33.441526    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:33.598965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:33.595182    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:33.945003 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:33.925265    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:45:33.945235 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:33.945070    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.135:15021/healthz/ready\": dial tcp 10.133.0.135:15021: connect: connection refused" start-of-body=
Oct 31 21:45:33.945235 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:33.945131    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm" podUID="3996ffdd-3dbe-41f6-ab61-62102a86e76b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.135:15021/healthz/ready\": dial tcp 10.133.0.135:15021: connect: connection refused"
Oct 31 21:45:34.936964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:34.931720    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.135:15021/healthz/ready\": dial tcp 10.133.0.135:15021: connect: connection refused" start-of-body=
Oct 31 21:45:34.936964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:34.931803    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm" podUID="3996ffdd-3dbe-41f6-ab61-62102a86e76b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.135:15021/healthz/ready\": dial tcp 10.133.0.135:15021: connect: connection refused"
Oct 31 21:45:35.362744 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:35.362706    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:35.939728 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:35.939680    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.135:15021/healthz/ready\": dial tcp 10.133.0.135:15021: connect: connection refused" start-of-body=
Oct 31 21:45:35.940272 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:35.939745    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm" podUID="3996ffdd-3dbe-41f6-ab61-62102a86e76b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.135:15021/healthz/ready\": dial tcp 10.133.0.135:15021: connect: connection refused"
Oct 31 21:45:36.944966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:36.944044    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.135:15021/healthz/ready\": dial tcp 10.133.0.135:15021: connect: connection refused" start-of-body=
Oct 31 21:45:36.944966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:36.944111    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm" podUID="3996ffdd-3dbe-41f6-ab61-62102a86e76b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.135:15021/healthz/ready\": dial tcp 10.133.0.135:15021: connect: connection refused"
Oct 31 21:45:37.252459 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:37.252371    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:37.370990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:37.370917    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:37.481353 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:37.481315    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:37.935357 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:37.935320    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.135:15021/healthz/ready\": dial tcp 10.133.0.135:15021: connect: connection refused" start-of-body=
Oct 31 21:45:37.935640 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:37.935610    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm" podUID="3996ffdd-3dbe-41f6-ab61-62102a86e76b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.135:15021/healthz/ready\": dial tcp 10.133.0.135:15021: connect: connection refused"
Oct 31 21:45:38.690281 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:38.690256    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:45:38.774903 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:38.774852    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istiod-ca-cert\") pod \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") "
Oct 31 21:45:38.775155 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:38.774913    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-workload-certs\") pod \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") "
Oct 31 21:45:38.775155 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:38.775021    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-kube-api-access-5r9cl\" (UniqueName: \"kubernetes.io/projected/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-kube-api-access-5r9cl\") pod \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") "
Oct 31 21:45:38.775155 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:38.775086    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-credential-socket\") pod \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") "
Oct 31 21:45:38.775689 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:38.775642    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-workload-certs" pod "19fcad1c-5c6e-4274-936d-bf93ca76c7d4" (UID: "19fcad1c-5c6e-4274-936d-bf93ca76c7d4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:45:38.776421 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:38.776383    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istiod-ca-cert" pod "19fcad1c-5c6e-4274-936d-bf93ca76c7d4" (UID: "19fcad1c-5c6e-4274-936d-bf93ca76c7d4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:45:38.776629 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:38.775299    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istio-podinfo\") pod \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") "
Oct 31 21:45:38.776912 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:38.776888    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-credential-socket" pod "19fcad1c-5c6e-4274-936d-bf93ca76c7d4" (UID: "19fcad1c-5c6e-4274-936d-bf93ca76c7d4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:45:38.777532 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:38.777232    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-workload-socket" pod "19fcad1c-5c6e-4274-936d-bf93ca76c7d4" (UID: "19fcad1c-5c6e-4274-936d-bf93ca76c7d4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:45:38.777643 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:38.776898    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-workload-socket\") pod \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") "
Oct 31 21:45:38.777779 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:38.777761    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istio-token\" (UniqueName: \"kubernetes.io/projected/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istio-token\") pod \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") "
Oct 31 21:45:38.777911 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:38.777894    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istio-envoy\") pod \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") "
Oct 31 21:45:38.778054 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:38.778037    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istio-data\") pod \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\" (UID: \"19fcad1c-5c6e-4274-936d-bf93ca76c7d4\") "
Oct 31 21:45:38.783424 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:38.783395    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:38.783532 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:38.783430    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:38.783532 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:38.783444    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:38.783532 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:38.783457    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:38.783694 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:38.783674    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istio-data" pod "19fcad1c-5c6e-4274-936d-bf93ca76c7d4" (UID: "19fcad1c-5c6e-4274-936d-bf93ca76c7d4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:45:38.786191 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:38.786161    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istio-envoy" pod "19fcad1c-5c6e-4274-936d-bf93ca76c7d4" (UID: "19fcad1c-5c6e-4274-936d-bf93ca76c7d4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:45:38.786305 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:38.786240    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istio-podinfo" pod "19fcad1c-5c6e-4274-936d-bf93ca76c7d4" (UID: "19fcad1c-5c6e-4274-936d-bf93ca76c7d4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:45:38.786450 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:38.786424    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-kube-api-access-5r9cl" pod "19fcad1c-5c6e-4274-936d-bf93ca76c7d4" (UID: "19fcad1c-5c6e-4274-936d-bf93ca76c7d4"). InnerVolumeSpecName "kube-api-access-5r9cl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:45:38.787418 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:38.787386    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istio-token" pod "19fcad1c-5c6e-4274-936d-bf93ca76c7d4" (UID: "19fcad1c-5c6e-4274-936d-bf93ca76c7d4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:45:38.884795 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:38.884760    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5r9cl\" (UniqueName: \"kubernetes.io/projected/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-kube-api-access-5r9cl\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:38.884795 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:38.884790    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:38.884795 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:38.884799    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:38.885027 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:38.884807    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:38.885027 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:38.884814    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19fcad1c-5c6e-4274-936d-bf93ca76c7d4-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:38.921975 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:38.921929    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.135:15021/healthz/ready\": dial tcp 10.133.0.135:15021: connect: connection refused" start-of-body=
Oct 31 21:45:38.922130 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:38.921999    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm" podUID="3996ffdd-3dbe-41f6-ab61-62102a86e76b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.135:15021/healthz/ready\": dial tcp 10.133.0.135:15021: connect: connection refused"
Oct 31 21:45:39.124611 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:39.124502    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:39.256251 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:39.256212    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:39.453011 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:39.452761    4460 generic.go:356] "Generic (PLEG): container finished" podID="19fcad1c-5c6e-4274-936d-bf93ca76c7d4" containerID="9b1af9a77e34f42b623c8e37da558b7140d452b72431ff1c949b7ceb52ec6fff" exitCode=0
Oct 31 21:45:39.453011 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:39.452973    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc" event={"ID":"19fcad1c-5c6e-4274-936d-bf93ca76c7d4","Type":"ContainerDied","Data":"9b1af9a77e34f42b623c8e37da558b7140d452b72431ff1c949b7ceb52ec6fff"}
Oct 31 21:45:39.453240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:39.453027    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc" event={"ID":"19fcad1c-5c6e-4274-936d-bf93ca76c7d4","Type":"ContainerDied","Data":"02bf264760c542bb6e531057b914e3652b7497e7b3906b93f26c7ab363b7087e"}
Oct 31 21:45:39.453240 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:39.453056    4460 scope.go:122] "RemoveContainer" containerID="9b1af9a77e34f42b623c8e37da558b7140d452b72431ff1c949b7ceb52ec6fff"
Oct 31 21:45:39.453349 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:39.453297    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"
Oct 31 21:45:39.474331 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:39.474289    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"]
Oct 31 21:45:39.474664 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:39.474609    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648" podUID="2b619644-e691-44de-889f-782d25668bca" containerName="istio-proxy" containerID="cri-o://82562daaec4145194690b8821b6db73e660c70e6f2deb0a4d263ee3d8ca1e687" gracePeriod=30
Oct 31 21:45:39.497842 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:39.497823    4460 scope.go:122] "RemoveContainer" containerID="2107324f230fcb14ac11586aff5b1c9a7693440ded3798f4910f5296f1c59b06"
Oct 31 21:45:39.505173 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:39.504332    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:39.512110 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:39.512079    4460 scope.go:122] "RemoveContainer" containerID="9b1af9a77e34f42b623c8e37da558b7140d452b72431ff1c949b7ceb52ec6fff"
Oct 31 21:45:39.512622 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:45:39.512591    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b1af9a77e34f42b623c8e37da558b7140d452b72431ff1c949b7ceb52ec6fff\": container with ID starting with 9b1af9a77e34f42b623c8e37da558b7140d452b72431ff1c949b7ceb52ec6fff not found: ID does not exist" containerID="9b1af9a77e34f42b623c8e37da558b7140d452b72431ff1c949b7ceb52ec6fff"
Oct 31 21:45:39.512725 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:39.512633    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b1af9a77e34f42b623c8e37da558b7140d452b72431ff1c949b7ceb52ec6fff"} err="failed to get container status \"9b1af9a77e34f42b623c8e37da558b7140d452b72431ff1c949b7ceb52ec6fff\": rpc error: code = NotFound desc = could not find container \"9b1af9a77e34f42b623c8e37da558b7140d452b72431ff1c949b7ceb52ec6fff\": container with ID starting with 9b1af9a77e34f42b623c8e37da558b7140d452b72431ff1c949b7ceb52ec6fff not found: ID does not exist"
Oct 31 21:45:39.512725 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:39.512656    4460 scope.go:122] "RemoveContainer" containerID="2107324f230fcb14ac11586aff5b1c9a7693440ded3798f4910f5296f1c59b06"
Oct 31 21:45:39.512991 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:45:39.512964    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2107324f230fcb14ac11586aff5b1c9a7693440ded3798f4910f5296f1c59b06\": container with ID starting with 2107324f230fcb14ac11586aff5b1c9a7693440ded3798f4910f5296f1c59b06 not found: ID does not exist" containerID="2107324f230fcb14ac11586aff5b1c9a7693440ded3798f4910f5296f1c59b06"
Oct 31 21:45:39.513149 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:39.513123    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2107324f230fcb14ac11586aff5b1c9a7693440ded3798f4910f5296f1c59b06"} err="failed to get container status \"2107324f230fcb14ac11586aff5b1c9a7693440ded3798f4910f5296f1c59b06\": rpc error: code = NotFound desc = could not find container \"2107324f230fcb14ac11586aff5b1c9a7693440ded3798f4910f5296f1c59b06\": container with ID starting with 2107324f230fcb14ac11586aff5b1c9a7693440ded3798f4910f5296f1c59b06 not found: ID does not exist"
Oct 31 21:45:39.566738 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:39.566705    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"]
Oct 31 21:45:39.570826 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:39.570797    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-b-f4569e-openshift-default-544c6f5449-cfxnc"]
Oct 31 21:45:40.228417 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:40.228379    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="19fcad1c-5c6e-4274-936d-bf93ca76c7d4" path="/var/lib/kubelet/pods/19fcad1c-5c6e-4274-936d-bf93ca76c7d4/volumes"
Oct 31 21:45:40.265593 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:40.265559    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:45:40.266106 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:40.266084    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:45:40.267150 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:40.267132    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:45:41.108535 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:41.108502    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:42.942306 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:42.942269    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:44.619269 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:44.619233    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:44.829816 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:44.829791    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:45:44.912408 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:44.912374    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2b619644-e691-44de-889f-782d25668bca-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b619644-e691-44de-889f-782d25668bca-istio-podinfo\") pod \"2b619644-e691-44de-889f-782d25668bca\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") "
Oct 31 21:45:44.912587 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:44.912427    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2b619644-e691-44de-889f-782d25668bca-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b619644-e691-44de-889f-782d25668bca-istiod-ca-cert\") pod \"2b619644-e691-44de-889f-782d25668bca\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") "
Oct 31 21:45:44.912587 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:44.912545    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-workload-socket\") pod \"2b619644-e691-44de-889f-782d25668bca\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") "
Oct 31 21:45:44.912721 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:44.912624    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-istio-data\") pod \"2b619644-e691-44de-889f-782d25668bca\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") "
Oct 31 21:45:44.912721 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:44.912673    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-istio-envoy\") pod \"2b619644-e691-44de-889f-782d25668bca\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") "
Oct 31 21:45:44.912721 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:44.912706    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b619644-e691-44de-889f-782d25668bca-istio-token\" (UniqueName: \"kubernetes.io/projected/2b619644-e691-44de-889f-782d25668bca-istio-token\") pod \"2b619644-e691-44de-889f-782d25668bca\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") "
Oct 31 21:45:44.912879 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:44.912800    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-credential-socket\") pod \"2b619644-e691-44de-889f-782d25668bca\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") "
Oct 31 21:45:44.912879 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:44.912867    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-workload-certs\") pod \"2b619644-e691-44de-889f-782d25668bca\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") "
Oct 31 21:45:44.913007 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:44.912924    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b619644-e691-44de-889f-782d25668bca-istiod-ca-cert" pod "2b619644-e691-44de-889f-782d25668bca" (UID: "2b619644-e691-44de-889f-782d25668bca"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:45:44.913007 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:44.912965    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b619644-e691-44de-889f-782d25668bca-kube-api-access-ppt9g\" (UniqueName: \"kubernetes.io/projected/2b619644-e691-44de-889f-782d25668bca-kube-api-access-ppt9g\") pod \"2b619644-e691-44de-889f-782d25668bca\" (UID: \"2b619644-e691-44de-889f-782d25668bca\") "
Oct 31 21:45:44.913302 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:44.913233    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-credential-socket" pod "2b619644-e691-44de-889f-782d25668bca" (UID: "2b619644-e691-44de-889f-782d25668bca"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:45:44.913515 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:44.913456    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-istio-data" pod "2b619644-e691-44de-889f-782d25668bca" (UID: "2b619644-e691-44de-889f-782d25668bca"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:45:44.913515 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:44.913479    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-workload-certs" pod "2b619644-e691-44de-889f-782d25668bca" (UID: "2b619644-e691-44de-889f-782d25668bca"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:45:44.913671 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:44.913576    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-workload-socket" pod "2b619644-e691-44de-889f-782d25668bca" (UID: "2b619644-e691-44de-889f-782d25668bca"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:45:44.913760 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:44.913724    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:44.913760 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:44.913754    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:44.913872 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:44.913771    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b619644-e691-44de-889f-782d25668bca-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:44.913872 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:44.913783    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:44.913872 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:44.913797    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:44.914988 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:44.914912    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2b619644-e691-44de-889f-782d25668bca-istio-podinfo" pod "2b619644-e691-44de-889f-782d25668bca" (UID: "2b619644-e691-44de-889f-782d25668bca"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:45:44.915645 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:44.915621    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b619644-e691-44de-889f-782d25668bca-kube-api-access-ppt9g" pod "2b619644-e691-44de-889f-782d25668bca" (UID: "2b619644-e691-44de-889f-782d25668bca"). InnerVolumeSpecName "kube-api-access-ppt9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:45:44.915753 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:44.915730    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-istio-envoy" pod "2b619644-e691-44de-889f-782d25668bca" (UID: "2b619644-e691-44de-889f-782d25668bca"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:45:44.916111 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:44.916091    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b619644-e691-44de-889f-782d25668bca-istio-token" pod "2b619644-e691-44de-889f-782d25668bca" (UID: "2b619644-e691-44de-889f-782d25668bca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:45:45.015188 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:45.015148    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b619644-e691-44de-889f-782d25668bca-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:45.015188 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:45.015190    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b619644-e691-44de-889f-782d25668bca-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:45.015369 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:45.015204    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppt9g\" (UniqueName: \"kubernetes.io/projected/2b619644-e691-44de-889f-782d25668bca-kube-api-access-ppt9g\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:45.015369 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:45.015216    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b619644-e691-44de-889f-782d25668bca-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:45:45.499130 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:45.499091    4460 generic.go:356] "Generic (PLEG): container finished" podID="2b619644-e691-44de-889f-782d25668bca" containerID="82562daaec4145194690b8821b6db73e660c70e6f2deb0a4d263ee3d8ca1e687" exitCode=0
Oct 31 21:45:45.499343 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:45.499138    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"
Oct 31 21:45:45.499343 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:45.499186    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648" event={"ID":"2b619644-e691-44de-889f-782d25668bca","Type":"ContainerDied","Data":"82562daaec4145194690b8821b6db73e660c70e6f2deb0a4d263ee3d8ca1e687"}
Oct 31 21:45:45.499343 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:45.499243    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648" event={"ID":"2b619644-e691-44de-889f-782d25668bca","Type":"ContainerDied","Data":"f530cc25556359a712144bdc70f3fe1989d69ad833f2354c808bdff83f94ea56"}
Oct 31 21:45:45.499343 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:45.499268    4460 scope.go:122] "RemoveContainer" containerID="82562daaec4145194690b8821b6db73e660c70e6f2deb0a4d263ee3d8ca1e687"
Oct 31 21:45:45.511110 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:45.511019    4460 scope.go:122] "RemoveContainer" containerID="ffe58be85d96ec3e16e38eb96c5ea214b280066966d7ebff1efe09da52ed886c"
Oct 31 21:45:45.520432 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:45.520413    4460 scope.go:122] "RemoveContainer" containerID="82562daaec4145194690b8821b6db73e660c70e6f2deb0a4d263ee3d8ca1e687"
Oct 31 21:45:45.520681 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:45:45.520660    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82562daaec4145194690b8821b6db73e660c70e6f2deb0a4d263ee3d8ca1e687\": container with ID starting with 82562daaec4145194690b8821b6db73e660c70e6f2deb0a4d263ee3d8ca1e687 not found: ID does not exist" containerID="82562daaec4145194690b8821b6db73e660c70e6f2deb0a4d263ee3d8ca1e687"
Oct 31 21:45:45.520761 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:45.520688    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82562daaec4145194690b8821b6db73e660c70e6f2deb0a4d263ee3d8ca1e687"} err="failed to get container status \"82562daaec4145194690b8821b6db73e660c70e6f2deb0a4d263ee3d8ca1e687\": rpc error: code = NotFound desc = could not find container \"82562daaec4145194690b8821b6db73e660c70e6f2deb0a4d263ee3d8ca1e687\": container with ID starting with 82562daaec4145194690b8821b6db73e660c70e6f2deb0a4d263ee3d8ca1e687 not found: ID does not exist"
Oct 31 21:45:45.520761 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:45.520706    4460 scope.go:122] "RemoveContainer" containerID="ffe58be85d96ec3e16e38eb96c5ea214b280066966d7ebff1efe09da52ed886c"
Oct 31 21:45:45.520926 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:45:45.520909    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffe58be85d96ec3e16e38eb96c5ea214b280066966d7ebff1efe09da52ed886c\": container with ID starting with ffe58be85d96ec3e16e38eb96c5ea214b280066966d7ebff1efe09da52ed886c not found: ID does not exist" containerID="ffe58be85d96ec3e16e38eb96c5ea214b280066966d7ebff1efe09da52ed886c"
Oct 31 21:45:45.521001 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:45.520931    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffe58be85d96ec3e16e38eb96c5ea214b280066966d7ebff1efe09da52ed886c"} err="failed to get container status \"ffe58be85d96ec3e16e38eb96c5ea214b280066966d7ebff1efe09da52ed886c\": rpc error: code = NotFound desc = could not find container \"ffe58be85d96ec3e16e38eb96c5ea214b280066966d7ebff1efe09da52ed886c\": container with ID starting with ffe58be85d96ec3e16e38eb96c5ea214b280066966d7ebff1efe09da52ed886c not found: ID does not exist"
Oct 31 21:45:45.573609 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:45.573581    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"]
Oct 31 21:45:45.587261 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:45.587237    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-d43b66-openshift-default-6b4c8f7cb9-n7648"]
Oct 31 21:45:46.225638 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.225595    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2b619644-e691-44de-889f-782d25668bca" path="/var/lib/kubelet/pods/2b619644-e691-44de-889f-782d25668bca/volumes"
Oct 31 21:45:46.318387 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.318353    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:46.605688 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.605651    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"]
Oct 31 21:45:46.606397 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.606375    4460 cpu_manager.go:414] "containerMap: removing container" podUID="2b619644-e691-44de-889f-782d25668bca" containerName="istio-proxy"
Oct 31 21:45:46.606503 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.606400    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b619644-e691-44de-889f-782d25668bca" containerName="istio-proxy"
Oct 31 21:45:46.606503 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.606430    4460 cpu_manager.go:414] "containerMap: removing container" podUID="19fcad1c-5c6e-4274-936d-bf93ca76c7d4" containerName="istio-proxy"
Oct 31 21:45:46.606503 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.606438    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19fcad1c-5c6e-4274-936d-bf93ca76c7d4" containerName="istio-proxy"
Oct 31 21:45:46.606503 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.606452    4460 cpu_manager.go:414] "containerMap: removing container" podUID="6c9f8306-0024-4639-8cf8-7e2b0a78e033" containerName="cleanup"
Oct 31 21:45:46.606503 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.606457    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6c9f8306-0024-4639-8cf8-7e2b0a78e033" containerName="cleanup"
Oct 31 21:45:46.606503 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.606463    4460 cpu_manager.go:414] "containerMap: removing container" podUID="19fcad1c-5c6e-4274-936d-bf93ca76c7d4" containerName="istio-proxy"
Oct 31 21:45:46.606503 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.606468    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19fcad1c-5c6e-4274-936d-bf93ca76c7d4" containerName="istio-proxy"
Oct 31 21:45:46.606503 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.606478    4460 cpu_manager.go:414] "containerMap: removing container" podUID="2b619644-e691-44de-889f-782d25668bca" containerName="istio-proxy"
Oct 31 21:45:46.606503 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.606483    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b619644-e691-44de-889f-782d25668bca" containerName="istio-proxy"
Oct 31 21:45:46.606811 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.606607    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="575790f7-e8f3-49bb-8d95-60f745bc8cff" containerName="cleanup"
Oct 31 21:45:46.606811 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.606622    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="9973b2ec-4365-42d4-98c2-f10cea058b4d" containerName="cleanup"
Oct 31 21:45:46.606811 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.606633    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b619644-e691-44de-889f-782d25668bca" containerName="istio-proxy"
Oct 31 21:45:46.606811 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.606643    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="19fcad1c-5c6e-4274-936d-bf93ca76c7d4" containerName="istio-proxy"
Oct 31 21:45:46.606811 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.606650    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="6c9f8306-0024-4639-8cf8-7e2b0a78e033" containerName="cleanup"
Oct 31 21:45:46.606811 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.606656    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="19fcad1c-5c6e-4274-936d-bf93ca76c7d4" containerName="istio-proxy"
Oct 31 21:45:46.606811 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.606662    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b619644-e691-44de-889f-782d25668bca" containerName="istio-proxy"
Oct 31 21:45:46.611996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.611926    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:45:46.625309 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.625274    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"]
Oct 31 21:45:46.630257 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.630235    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:45:46.638821 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.636810    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"]
Oct 31 21:45:46.656070 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.656040    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"]
Oct 31 21:45:46.734838 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.734795    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-credential-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:45:46.734838 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.734831    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t4t4\" (UniqueName: \"kubernetes.io/projected/96a8801d-ad9e-4268-8dd0-a657e235e83c-kube-api-access-5t4t4\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:45:46.735073 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.734913    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-workload-certs\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:45:46.735073 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.735043    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56d5066a-9abc-4b3d-b905-2250ef807321-istio-podinfo\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:45:46.735193 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.735080    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96a8801d-ad9e-4268-8dd0-a657e235e83c-istio-podinfo\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:45:46.735193 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.735168    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96a8801d-ad9e-4268-8dd0-a657e235e83c-istiod-ca-cert\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:45:46.735309 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.735220    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-istio-envoy\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:45:46.735309 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.735284    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56d5066a-9abc-4b3d-b905-2250ef807321-istiod-ca-cert\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:45:46.735385 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.735333    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-istio-data\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:45:46.735385 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.735369    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-workload-certs\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:45:46.735474 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.735390    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56d5066a-9abc-4b3d-b905-2250ef807321-istio-token\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:45:46.735474 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.735423    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96a8801d-ad9e-4268-8dd0-a657e235e83c-istio-token\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:45:46.735595 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.735571    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nshdr\" (UniqueName: \"kubernetes.io/projected/56d5066a-9abc-4b3d-b905-2250ef807321-kube-api-access-nshdr\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:45:46.735698 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.735685    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-workload-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:45:46.735735 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.735708    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-istio-data\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:45:46.735735 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.735725    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-credential-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:45:46.735836 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.735796    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-istio-envoy\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:45:46.735894 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.735837    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-workload-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:45:46.837151 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.837114    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-workload-certs\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:45:46.837347 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.837188    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56d5066a-9abc-4b3d-b905-2250ef807321-istio-podinfo\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:45:46.837347 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.837205    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96a8801d-ad9e-4268-8dd0-a657e235e83c-istio-podinfo\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:45:46.837347 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.837246    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96a8801d-ad9e-4268-8dd0-a657e235e83c-istiod-ca-cert\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:45:46.837347 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.837269    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-istio-envoy\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:45:46.837347 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.837297    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56d5066a-9abc-4b3d-b905-2250ef807321-istiod-ca-cert\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:45:46.837347 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.837332    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-istio-data\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:45:46.837680 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.837439    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-workload-certs\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:45:46.837680 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.837485    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56d5066a-9abc-4b3d-b905-2250ef807321-istio-token\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:45:46.837680 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.837518    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96a8801d-ad9e-4268-8dd0-a657e235e83c-istio-token\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:45:46.837680 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.837597    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-workload-certs\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:45:46.837680 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.837619    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nshdr\" (UniqueName: \"kubernetes.io/projected/56d5066a-9abc-4b3d-b905-2250ef807321-kube-api-access-nshdr\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:45:46.837916 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.837702    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-istio-data\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:45:46.837916 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.837764    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-workload-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:45:46.837916 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.837798    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-istio-data\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:45:46.837916 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.837828    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-credential-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:45:46.837916 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.837900    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-istio-envoy\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:45:46.838243 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.837979    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-workload-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:45:46.838243 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.838009    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-credential-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:45:46.838243 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.838039    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t4t4\" (UniqueName: \"kubernetes.io/projected/96a8801d-ad9e-4268-8dd0-a657e235e83c-kube-api-access-5t4t4\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:45:46.838243 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.838119    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96a8801d-ad9e-4268-8dd0-a657e235e83c-istiod-ca-cert\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:45:46.838243 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.838148    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56d5066a-9abc-4b3d-b905-2250ef807321-istiod-ca-cert\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:45:46.838506 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.838424    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-credential-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:45:46.838506 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.838444    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-workload-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:45:46.838637 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.838515    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-istio-data\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:45:46.838637 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.838548    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-workload-certs\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:45:46.838747 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.838722    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-workload-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:45:46.838805 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.838744    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-credential-socket\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:45:46.839920 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.839890    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-istio-envoy\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:45:46.840076 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.840057    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56d5066a-9abc-4b3d-b905-2250ef807321-istio-podinfo\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:45:46.840346 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.840330    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-istio-envoy\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:45:46.840474 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.840455    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96a8801d-ad9e-4268-8dd0-a657e235e83c-istio-podinfo\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:45:46.862795 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.862727    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56d5066a-9abc-4b3d-b905-2250ef807321-istio-token\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:45:46.862930 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.862905    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t4t4\" (UniqueName: \"kubernetes.io/projected/96a8801d-ad9e-4268-8dd0-a657e235e83c-kube-api-access-5t4t4\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:45:46.863116 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.863099    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nshdr\" (UniqueName: \"kubernetes.io/projected/56d5066a-9abc-4b3d-b905-2250ef807321-kube-api-access-nshdr\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:45:46.863502 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.863483    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96a8801d-ad9e-4268-8dd0-a657e235e83c-istio-token\") pod \"e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") " pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:45:46.938711 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.938687    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:45:46.953906 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:46.953880    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:45:47.140970 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:47.140877    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:47.274199 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:47.272095    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"]
Oct 31 21:45:47.278912 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:47.278864    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:45:47.279089 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:47.279038    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:45:47.279181 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:47.279161    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:45:47.282976 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:47.282952    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"]
Oct 31 21:45:47.292820 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:47.292793    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:45:47.292901 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:47.292849    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:45:47.292901 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:47.292877    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:45:47.517263 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:47.517231    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm" event={"ID":"56d5066a-9abc-4b3d-b905-2250ef807321","Type":"ContainerStarted","Data":"c0b783b082ea28c91ea5400013bbbb4ca732339f5eee490598fd577660a3b4d5"}
Oct 31 21:45:47.517452 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:47.517293    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm" event={"ID":"56d5066a-9abc-4b3d-b905-2250ef807321","Type":"ContainerStarted","Data":"d60fbf9340d5474808cb55271ee84fe0c1ce9f72a5f8f7984546541ace0f2ad2"}
Oct 31 21:45:47.519118 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:47.519085    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g" event={"ID":"96a8801d-ad9e-4268-8dd0-a657e235e83c","Type":"ContainerStarted","Data":"0c63ce52d24a8104077be1e4117001078753b5160412fecf19fc55adde974036"}
Oct 31 21:45:47.519258 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:47.519138    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g" event={"ID":"96a8801d-ad9e-4268-8dd0-a657e235e83c","Type":"ContainerStarted","Data":"68b11d2dbde9c21f6197d562bf6d9139e611428085704a6fca038957298c9a75"}
Oct 31 21:45:47.566704 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:47.566649    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm" podStartSLOduration=1.566633196 podStartE2EDuration="1.566633196s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:45:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:45:47.565136806 +0000 UTC m=+1787.481446236" watchObservedRunningTime="2026-10-31 21:45:47.566633196 +0000 UTC m=+1787.482942662"
Oct 31 21:45:47.616845 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:47.616782    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g" podStartSLOduration=1.6167640859999999 podStartE2EDuration="1.616764086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:45:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:45:47.612806946 +0000 UTC m=+1787.529116367" watchObservedRunningTime="2026-10-31 21:45:47.616764086 +0000 UTC m=+1787.533073515"
Oct 31 21:45:47.938884 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:47.938800    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:45:47.953995 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:47.953966    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:45:48.180964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:48.172956    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:48.966971 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:48.965431    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused" start-of-body=
Oct 31 21:45:48.966971 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:48.965514    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm" podUID="56d5066a-9abc-4b3d-b905-2250ef807321" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused"
Oct 31 21:45:48.975305 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:48.975170    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused" start-of-body=
Oct 31 21:45:48.975305 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:48.975224    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g" podUID="96a8801d-ad9e-4268-8dd0-a657e235e83c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused"
Oct 31 21:45:49.942409 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:49.942369    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused" start-of-body=
Oct 31 21:45:49.942725 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:49.942692    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm" podUID="56d5066a-9abc-4b3d-b905-2250ef807321" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused"
Oct 31 21:45:49.959319 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:49.959286    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused" start-of-body=
Oct 31 21:45:49.959548 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:49.959518    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g" podUID="96a8801d-ad9e-4268-8dd0-a657e235e83c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused"
Oct 31 21:45:50.571453 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:50.571408    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:50.950019 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:50.943344    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused" start-of-body=
Oct 31 21:45:50.950019 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:50.943407    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm" podUID="56d5066a-9abc-4b3d-b905-2250ef807321" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused"
Oct 31 21:45:50.959963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:50.955627    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused" start-of-body=
Oct 31 21:45:50.959963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:50.955674    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g" podUID="96a8801d-ad9e-4268-8dd0-a657e235e83c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused"
Oct 31 21:45:51.494319 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:45:51.494278    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:45:51.944964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:51.943525    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused" start-of-body=
Oct 31 21:45:51.944964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:51.943590    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm" podUID="56d5066a-9abc-4b3d-b905-2250ef807321" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused"
Oct 31 21:45:51.956972 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:51.956344    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused" start-of-body=
Oct 31 21:45:51.956972 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:51.956400    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g" podUID="96a8801d-ad9e-4268-8dd0-a657e235e83c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused"
Oct 31 21:45:52.953965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:52.947410    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused" start-of-body=
Oct 31 21:45:52.953965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:52.947484    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm" podUID="56d5066a-9abc-4b3d-b905-2250ef807321" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused"
Oct 31 21:45:52.961968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:52.958322    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused" start-of-body=
Oct 31 21:45:52.961968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:52.958384    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g" podUID="96a8801d-ad9e-4268-8dd0-a657e235e83c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused"
Oct 31 21:45:52.991151 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:52.991117    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:53.939464 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:53.939430    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused" start-of-body=
Oct 31 21:45:53.939702 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:53.939485    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm" podUID="56d5066a-9abc-4b3d-b905-2250ef807321" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused"
Oct 31 21:45:53.954637 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:53.954599    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused" start-of-body=
Oct 31 21:45:53.955111 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:53.954663    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g" podUID="96a8801d-ad9e-4268-8dd0-a657e235e83c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.137:15021/healthz/ready\": dial tcp 10.133.0.137:15021: connect: connection refused"
Oct 31 21:45:54.126315 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:54.126269    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:54.958715 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:54.958682    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:45:54.959191 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:54.959175    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:45:54.959983 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:54.959927    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:45:54.986151 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:54.986119    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:45:54.986467 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:54.986435    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:45:54.988125 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:54.988102    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:45:55.107848 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:55.107814    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:45:55.832673 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:55.832632    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:57.978699 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:57.978661    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:45:58.127075 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:58.127030    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh" podUID="869ed893-352c-4f8f-a174-b42afdeb4de7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.77:8000/ready\": context deadline exceeded"
Oct 31 21:45:58.173118 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:58.171824    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747" podUID="d1078aa7-1947-45ce-b6bb-d6550ad61fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 31 21:45:59.742626 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:45:59.742582    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:00.559265 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:00.559238    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ssl9_26069598-5dc4-4c96-b55f-dd2f09884419/ovn-acl-logging/0.log"
Oct 31 21:46:00.565221 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:00.565200    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ssl9_26069598-5dc4-4c96-b55f-dd2f09884419/ovn-acl-logging/0.log"
Oct 31 21:46:01.489388 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:01.489351    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:03.230096 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:03.230058    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:04.777386 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:04.777350    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:05.021063 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:05.021025    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:06.943899 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:06.943855    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:08.756048 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:08.756013    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:10.552965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:10.533893    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:12.294865 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:12.294730    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:12.301131 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:12.300888    4460 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" podUID="af7e0e0c-cf52-4270-b528-35718339996c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:46:12.434112 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:12.433931    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:12.549432 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:12.549343    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:12.889572 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:12.889494    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:13.005158 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:13.005123    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:13.113393 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:13.113358    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:13.534416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:13.534380    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:14.927594 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:14.927549    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:16.710141 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:16.710104    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:16.978162 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:16.978066    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-448b01-6778f95d9-mjvcz"]
Oct 31 21:46:17.013808 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:17.013780    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-448b01-67f46999c5-jlp47"]
Oct 31 21:46:17.134383 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:17.134345    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:17.257028 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:17.251983    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:18.497921 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:18.497891    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:19.845428 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:19.845393    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:19.960965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:19.960451    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:20.169013 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:20.167505    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:20.284197 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:20.284155    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:21.452563 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:46:21.452530    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:46:21.669804 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:21.669764    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:21.848663 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:21.848628    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"]
Oct 31 21:46:21.854833 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:21.854804    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:46:21.882151 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:21.882120    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"]
Oct 31 21:46:21.954364 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:21.954329    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqlzn\" (UniqueName: \"kubernetes.io/projected/6870d753-72cd-409c-947b-442703ac27d0-kube-api-access-cqlzn\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:46:21.954534 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:21.954433    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6870d753-72cd-409c-947b-442703ac27d0-istiod-ca-cert\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:46:21.954534 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:21.954480    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-istio-data\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:46:21.954611 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:21.954559    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6870d753-72cd-409c-947b-442703ac27d0-istio-token\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:46:21.954611 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:21.954597    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-workload-certs\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:46:21.954722 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:21.954707    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-credential-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:46:21.954775 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:21.954732    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-istio-envoy\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:46:21.954917 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:21.954888    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-workload-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:46:21.955343 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:21.955095    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6870d753-72cd-409c-947b-442703ac27d0-istio-podinfo\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:46:21.974332 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:21.974305    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:21.979088 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:21.979049    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-praxis-a-448b01-6778f95d9-mjvcz" podUID="2113003b-165f-4533-aff5-85c75433f325" containerName="payload-processing" containerID="cri-o://2cc74fc21290b71d3c1aac8a2c8cfdfcb0e4dc149df3babdf8fd9f5be0971374" gracePeriod=25
Oct 31 21:46:22.014472 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.014441    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-448b01-67f46999c5-jlp47" podUID="bad71d55-48bf-4120-affb-c627a9988e3e" containerName="payload-pre-processing" containerID="cri-o://ee62bfe19404a037801a04e1a156dbb87eb539ad020d40fda901089063eb7b02" gracePeriod=25
Oct 31 21:46:22.056229 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.056201    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-workload-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:46:22.056359 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.056255    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6870d753-72cd-409c-947b-442703ac27d0-istio-podinfo\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:46:22.056359 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.056294    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqlzn\" (UniqueName: \"kubernetes.io/projected/6870d753-72cd-409c-947b-442703ac27d0-kube-api-access-cqlzn\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:46:22.056359 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.056331    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6870d753-72cd-409c-947b-442703ac27d0-istiod-ca-cert\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:46:22.056516 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.056366    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-istio-data\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:46:22.056516 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.056396    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6870d753-72cd-409c-947b-442703ac27d0-istio-token\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:46:22.056635 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.056509    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-workload-certs\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:46:22.056635 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.056590    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-credential-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:46:22.056635 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.056618    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-istio-envoy\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:46:22.056786 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.056639    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-workload-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:46:22.056786 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.056693    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-istio-data\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:46:22.057044 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.057019    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-credential-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:46:22.057223 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.057183    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-workload-certs\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:46:22.057420 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.057400    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6870d753-72cd-409c-947b-442703ac27d0-istiod-ca-cert\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:46:22.058731 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.058712    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-istio-envoy\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:46:22.058935 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.058914    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6870d753-72cd-409c-947b-442703ac27d0-istio-podinfo\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:46:22.081258 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.081229    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6870d753-72cd-409c-947b-442703ac27d0-istio-token\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:46:22.081344 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.081325    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqlzn\" (UniqueName: \"kubernetes.io/projected/6870d753-72cd-409c-947b-442703ac27d0-kube-api-access-cqlzn\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:46:22.180884 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.179296    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:46:22.180884 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.180164    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:22.307811 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.307780    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:22.308717 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.308683    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-448b01-6778f95d9-mjvcz"
Oct 31 21:46:22.520247 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.517992    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2113003b-165f-4533-aff5-85c75433f325-kube-api-access-f9t9t\" (UniqueName: \"kubernetes.io/projected/2113003b-165f-4533-aff5-85c75433f325-kube-api-access-f9t9t\") pod \"2113003b-165f-4533-aff5-85c75433f325\" (UID: \"2113003b-165f-4533-aff5-85c75433f325\") "
Oct 31 21:46:22.520247 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.518070    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2113003b-165f-4533-aff5-85c75433f325-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2113003b-165f-4533-aff5-85c75433f325-plugins-config-volume\") pod \"2113003b-165f-4533-aff5-85c75433f325\" (UID: \"2113003b-165f-4533-aff5-85c75433f325\") "
Oct 31 21:46:22.520247 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.520202    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2113003b-165f-4533-aff5-85c75433f325-plugins-config-volume" pod "2113003b-165f-4533-aff5-85c75433f325" (UID: "2113003b-165f-4533-aff5-85c75433f325"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:46:22.559764 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.559716    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2113003b-165f-4533-aff5-85c75433f325-kube-api-access-f9t9t" pod "2113003b-165f-4533-aff5-85c75433f325" (UID: "2113003b-165f-4533-aff5-85c75433f325"). InnerVolumeSpecName "kube-api-access-f9t9t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:46:22.567980 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.567956    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-448b01-67f46999c5-jlp47"
Oct 31 21:46:22.619922 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.619891    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bad71d55-48bf-4120-affb-c627a9988e3e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bad71d55-48bf-4120-affb-c627a9988e3e-plugins-config-volume\") pod \"bad71d55-48bf-4120-affb-c627a9988e3e\" (UID: \"bad71d55-48bf-4120-affb-c627a9988e3e\") "
Oct 31 21:46:22.620082 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.619933    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bad71d55-48bf-4120-affb-c627a9988e3e-kube-api-access-nsr4w\" (UniqueName: \"kubernetes.io/projected/bad71d55-48bf-4120-affb-c627a9988e3e-kube-api-access-nsr4w\") pod \"bad71d55-48bf-4120-affb-c627a9988e3e\" (UID: \"bad71d55-48bf-4120-affb-c627a9988e3e\") "
Oct 31 21:46:22.620238 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.620216    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bad71d55-48bf-4120-affb-c627a9988e3e-plugins-config-volume" pod "bad71d55-48bf-4120-affb-c627a9988e3e" (UID: "bad71d55-48bf-4120-affb-c627a9988e3e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:46:22.620486 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.620467    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9t9t\" (UniqueName: \"kubernetes.io/projected/2113003b-165f-4533-aff5-85c75433f325-kube-api-access-f9t9t\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:46:22.620572 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.620489    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bad71d55-48bf-4120-affb-c627a9988e3e-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:46:22.620572 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.620502    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2113003b-165f-4533-aff5-85c75433f325-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:46:22.622097 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.622073    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bad71d55-48bf-4120-affb-c627a9988e3e-kube-api-access-nsr4w" pod "bad71d55-48bf-4120-affb-c627a9988e3e" (UID: "bad71d55-48bf-4120-affb-c627a9988e3e"). InnerVolumeSpecName "kube-api-access-nsr4w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:46:22.721716 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.721684    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nsr4w\" (UniqueName: \"kubernetes.io/projected/bad71d55-48bf-4120-affb-c627a9988e3e-kube-api-access-nsr4w\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:46:22.764287 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.764259    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"]
Oct 31 21:46:22.765018 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:46:22.764992    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6870d753_72cd_409c_947b_442703ac27d0.slice/crio-b144f2cd36ae669b150c807ff5a50aba4aba66804545f71d8b6e2bb6ba85ed0c WatchSource:0}: Error finding container b144f2cd36ae669b150c807ff5a50aba4aba66804545f71d8b6e2bb6ba85ed0c: Status 404 returned error can't find the container with id b144f2cd36ae669b150c807ff5a50aba4aba66804545f71d8b6e2bb6ba85ed0c
Oct 31 21:46:22.767604 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.767573    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:46:22.767694 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.767635    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:46:22.767694 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:22.767667    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:46:23.022044 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:23.021967    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4" event={"ID":"6870d753-72cd-409c-947b-442703ac27d0","Type":"ContainerStarted","Data":"23fd0c583d448e7d048f473c59bd2615b895e141a3d476292873e3703cda20fd"}
Oct 31 21:46:23.022044 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:23.022023    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4" event={"ID":"6870d753-72cd-409c-947b-442703ac27d0","Type":"ContainerStarted","Data":"b144f2cd36ae669b150c807ff5a50aba4aba66804545f71d8b6e2bb6ba85ed0c"}
Oct 31 21:46:23.023398 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:23.023371    4460 generic.go:356] "Generic (PLEG): container finished" podID="bad71d55-48bf-4120-affb-c627a9988e3e" containerID="ee62bfe19404a037801a04e1a156dbb87eb539ad020d40fda901089063eb7b02" exitCode=0
Oct 31 21:46:23.023534 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:23.023409    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-448b01-67f46999c5-jlp47"
Oct 31 21:46:23.023534 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:23.023454    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-448b01-67f46999c5-jlp47" event={"ID":"bad71d55-48bf-4120-affb-c627a9988e3e","Type":"ContainerDied","Data":"ee62bfe19404a037801a04e1a156dbb87eb539ad020d40fda901089063eb7b02"}
Oct 31 21:46:23.023534 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:23.023503    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-448b01-67f46999c5-jlp47" event={"ID":"bad71d55-48bf-4120-affb-c627a9988e3e","Type":"ContainerDied","Data":"e1ddaebae4be7c3e05ad595b1e50a4aeaedfac2cf2a719aca59048164dcebf6d"}
Oct 31 21:46:23.023534 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:23.023532    4460 scope.go:122] "RemoveContainer" containerID="ee62bfe19404a037801a04e1a156dbb87eb539ad020d40fda901089063eb7b02"
Oct 31 21:46:23.025013 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:23.024991    4460 generic.go:356] "Generic (PLEG): container finished" podID="2113003b-165f-4533-aff5-85c75433f325" containerID="2cc74fc21290b71d3c1aac8a2c8cfdfcb0e4dc149df3babdf8fd9f5be0971374" exitCode=0
Oct 31 21:46:23.025120 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:23.025050    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-448b01-6778f95d9-mjvcz"
Oct 31 21:46:23.025120 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:23.025084    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-448b01-6778f95d9-mjvcz" event={"ID":"2113003b-165f-4533-aff5-85c75433f325","Type":"ContainerDied","Data":"2cc74fc21290b71d3c1aac8a2c8cfdfcb0e4dc149df3babdf8fd9f5be0971374"}
Oct 31 21:46:23.025199 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:23.025148    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-448b01-6778f95d9-mjvcz" event={"ID":"2113003b-165f-4533-aff5-85c75433f325","Type":"ContainerDied","Data":"b0854fb4ac31e27062bfcd077ef153bd9ea15d80158bf0d55a8c9b030b903c42"}
Oct 31 21:46:23.036440 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:23.036419    4460 scope.go:122] "RemoveContainer" containerID="ee62bfe19404a037801a04e1a156dbb87eb539ad020d40fda901089063eb7b02"
Oct 31 21:46:23.036716 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:46:23.036698    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee62bfe19404a037801a04e1a156dbb87eb539ad020d40fda901089063eb7b02\": container with ID starting with ee62bfe19404a037801a04e1a156dbb87eb539ad020d40fda901089063eb7b02 not found: ID does not exist" containerID="ee62bfe19404a037801a04e1a156dbb87eb539ad020d40fda901089063eb7b02"
Oct 31 21:46:23.036798 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:23.036730    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee62bfe19404a037801a04e1a156dbb87eb539ad020d40fda901089063eb7b02"} err="failed to get container status \"ee62bfe19404a037801a04e1a156dbb87eb539ad020d40fda901089063eb7b02\": rpc error: code = NotFound desc = could not find container \"ee62bfe19404a037801a04e1a156dbb87eb539ad020d40fda901089063eb7b02\": container with ID starting with ee62bfe19404a037801a04e1a156dbb87eb539ad020d40fda901089063eb7b02 not found: ID does not exist"
Oct 31 21:46:23.036798 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:23.036751    4460 scope.go:122] "RemoveContainer" containerID="2cc74fc21290b71d3c1aac8a2c8cfdfcb0e4dc149df3babdf8fd9f5be0971374"
Oct 31 21:46:23.047592 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:23.047577    4460 scope.go:122] "RemoveContainer" containerID="2cc74fc21290b71d3c1aac8a2c8cfdfcb0e4dc149df3babdf8fd9f5be0971374"
Oct 31 21:46:23.047837 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:46:23.047822    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cc74fc21290b71d3c1aac8a2c8cfdfcb0e4dc149df3babdf8fd9f5be0971374\": container with ID starting with 2cc74fc21290b71d3c1aac8a2c8cfdfcb0e4dc149df3babdf8fd9f5be0971374 not found: ID does not exist" containerID="2cc74fc21290b71d3c1aac8a2c8cfdfcb0e4dc149df3babdf8fd9f5be0971374"
Oct 31 21:46:23.047876 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:23.047844    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cc74fc21290b71d3c1aac8a2c8cfdfcb0e4dc149df3babdf8fd9f5be0971374"} err="failed to get container status \"2cc74fc21290b71d3c1aac8a2c8cfdfcb0e4dc149df3babdf8fd9f5be0971374\": rpc error: code = NotFound desc = could not find container \"2cc74fc21290b71d3c1aac8a2c8cfdfcb0e4dc149df3babdf8fd9f5be0971374\": container with ID starting with 2cc74fc21290b71d3c1aac8a2c8cfdfcb0e4dc149df3babdf8fd9f5be0971374 not found: ID does not exist"
Oct 31 21:46:23.076044 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:23.076002    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4" podStartSLOduration=2.075989606 podStartE2EDuration="2.075989606s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:46:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:46:23.072971018 +0000 UTC m=+1822.989280444" watchObservedRunningTime="2026-10-31 21:46:23.075989606 +0000 UTC m=+1822.992299034"
Oct 31 21:46:23.114830 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:23.114797    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-448b01-67f46999c5-jlp47"]
Oct 31 21:46:23.119097 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:23.119069    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-448b01-67f46999c5-jlp47"]
Oct 31 21:46:23.154451 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:23.154417    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-448b01-6778f95d9-mjvcz"]
Oct 31 21:46:23.158755 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:23.158733    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-448b01-6778f95d9-mjvcz"]
Oct 31 21:46:23.179754 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:23.179720    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:46:23.181457 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:23.181434    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused" start-of-body=
Oct 31 21:46:23.181551 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:23.181476    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4" podUID="6870d753-72cd-409c-947b-442703ac27d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused"
Oct 31 21:46:23.821290 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:23.821245    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-80542b-6c5b644b9-v8vqz"]
Oct 31 21:46:23.857080 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:23.856967    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:23.875905 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:23.875873    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-80542b-777f94c5f9-p4p27"]
Oct 31 21:46:23.990638 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:23.990604    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:24.116059 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:24.115973    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:24.185448 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:24.185394    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused" start-of-body=
Oct 31 21:46:24.185749 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:24.185718    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4" podUID="6870d753-72cd-409c-947b-442703ac27d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused"
Oct 31 21:46:24.256477 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:24.256447    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2113003b-165f-4533-aff5-85c75433f325" path="/var/lib/kubelet/pods/2113003b-165f-4533-aff5-85c75433f325/volumes"
Oct 31 21:46:24.257158 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:24.257140    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bad71d55-48bf-4120-affb-c627a9988e3e" path="/var/lib/kubelet/pods/bad71d55-48bf-4120-affb-c627a9988e3e/volumes"
Oct 31 21:46:25.180400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:25.180367    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused" start-of-body=
Oct 31 21:46:25.180807 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:25.180442    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4" podUID="6870d753-72cd-409c-947b-442703ac27d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused"
Oct 31 21:46:25.840514 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:25.840427    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:25.974134 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:25.974098    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:26.088872 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:26.088837    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:26.180341 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:26.180260    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused" start-of-body=
Oct 31 21:46:26.180341 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:26.180319    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4" podUID="6870d753-72cd-409c-947b-442703ac27d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused"
Oct 31 21:46:27.180285 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:27.180248    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused" start-of-body=
Oct 31 21:46:27.180556 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:27.180306    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4" podUID="6870d753-72cd-409c-947b-442703ac27d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused"
Oct 31 21:46:27.758007 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:27.757960    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:27.868387 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:27.868345    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:28.182174 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:28.182133    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused" start-of-body=
Oct 31 21:46:28.182792 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:28.182757    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4" podUID="6870d753-72cd-409c-947b-442703ac27d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused"
Oct 31 21:46:28.822618 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:28.822567    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-praxis-a-80542b-6c5b644b9-v8vqz" podUID="6868309a-8813-4420-bc7d-4a5b7a38eb46" containerName="payload-processing" containerID="cri-o://86558919e547044d38698f268e7b27e614a6089b58638b8067e36b00dfa84620" gracePeriod=25
Oct 31 21:46:28.877077 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:28.877006    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-80542b-777f94c5f9-p4p27" podUID="39007e3f-5368-4ae6-8d73-2e4d407cf76b" containerName="payload-pre-processing" containerID="cri-o://f7baeb0b935c7f80a1c418bb81b2b79ff14069fae064004451712dc26bf6ba24" gracePeriod=25
Oct 31 21:46:29.180818 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:29.180742    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused" start-of-body=
Oct 31 21:46:29.180818 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:29.180795    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4" podUID="6870d753-72cd-409c-947b-442703ac27d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused"
Oct 31 21:46:29.617928 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:29.617891    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:30.184932 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:30.184899    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:46:30.185242 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:30.185208    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:46:30.186022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:30.186003    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:46:31.269504 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:31.269471    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:32.943700 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:32.943667    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:33.163628 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:33.163591    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:34.832898 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:34.832868    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:34.996605 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:34.996570    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:46:35.111158 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:35.111090    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:35.222725 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:35.222696    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:35.464713 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:35.464611    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-80542b-777f94c5f9-p4p27" podUID="39007e3f-5368-4ae6-8d73-2e4d407cf76b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 21:46:35.464892 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:35.464708    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-praxis-a-80542b-6c5b644b9-v8vqz" podUID="6868309a-8813-4420-bc7d-4a5b7a38eb46" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 21:46:36.323003 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:36.322911    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:36.901670 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:36.901612    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:37.093978 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:37.093909    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:37.213023 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:37.208980    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh" podUID="869ed893-352c-4f8f-a174-b42afdeb4de7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:46:37.650965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:37.630987    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:37.746464 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:37.746427    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:38.150084 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:38.150042    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podUID="9cd67bf5-8833-4a82-bc33-bed7bdfd4d36" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:46:38.346205 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:38.346164    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747" podUID="d1078aa7-1947-45ce-b6bb-d6550ad61fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 31 21:46:38.388176 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:38.347455    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz" podUID="93a42128-ee74-4dcd-b158-27e33cfd44f9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": context deadline exceeded"
Oct 31 21:46:38.388911 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:38.362826    4460 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt" podUID="9771611f-879a-44f6-9c1a-f8647dc2d841" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/health\": context deadline exceeded"
Oct 31 21:46:38.389117 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:38.379507    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh" podUID="869ed893-352c-4f8f-a174-b42afdeb4de7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:46:38.389480 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:38.380874    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n" podUID="4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": context deadline exceeded"
Oct 31 21:46:38.408262 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:38.408188    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 21:46:38.531719 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:38.531678    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt" podUID="9771611f-879a-44f6-9c1a-f8647dc2d841" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:46:38.586174 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:38.586143    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 21:46:38.633610 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:38.627256    4460 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747" podUID="d1078aa7-1947-45ce-b6bb-d6550ad61fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:46:38.833143 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:38.833102    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" podUID="af7e0e0c-cf52-4270-b528-35718339996c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:46:39.846471 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:39.846437    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:41.615536 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:41.615503    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:43.297189 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:43.297084    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:43.906002 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:43.905932    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:44.016045 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:44.016011    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:44.228580 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:44.228491    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"]
Oct 31 21:46:44.228750 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:44.228685    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns" podUID="78bb7115-5485-45ec-9b35-a76a75ba3eff" containerName="istio-proxy" containerID="cri-o://41d8d0da12535ca5380761cf68aa470090d3a575d8ae88a7af62410240b8ddd9" gracePeriod=30
Oct 31 21:46:44.250792 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:44.250756    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:45.173818 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:45.173778    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:45.285346 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:45.285308    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:45.399039 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:45.399000    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:45.464620 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:45.464517    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-80542b-777f94c5f9-p4p27" podUID="39007e3f-5368-4ae6-8d73-2e4d407cf76b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 21:46:45.464620 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:45.464598    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-praxis-a-80542b-6c5b644b9-v8vqz" podUID="6868309a-8813-4420-bc7d-4a5b7a38eb46" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 21:46:47.009077 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:47.009041    4460 patch_prober.go:29] interesting pod/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 21:46:47.009455 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:47.009106    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns" podUID="78bb7115-5485-45ec-9b35-a76a75ba3eff" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 21:46:47.056740 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:47.056701    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:47.180476 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:47.180439    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:47.296137 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:47.296102    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:48.814586 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:48.814552    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:49.151080 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.151050    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-80542b-6c5b644b9-v8vqz"
Oct 31 21:46:49.210118 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.210086    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-80542b-777f94c5f9-p4p27"
Oct 31 21:46:49.234977 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.234926    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6868309a-8813-4420-bc7d-4a5b7a38eb46-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6868309a-8813-4420-bc7d-4a5b7a38eb46-plugins-config-volume\") pod \"6868309a-8813-4420-bc7d-4a5b7a38eb46\" (UID: \"6868309a-8813-4420-bc7d-4a5b7a38eb46\") "
Oct 31 21:46:49.235290 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.235261    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6868309a-8813-4420-bc7d-4a5b7a38eb46-kube-api-access-n2ml2\" (UniqueName: \"kubernetes.io/projected/6868309a-8813-4420-bc7d-4a5b7a38eb46-kube-api-access-n2ml2\") pod \"6868309a-8813-4420-bc7d-4a5b7a38eb46\" (UID: \"6868309a-8813-4420-bc7d-4a5b7a38eb46\") "
Oct 31 21:46:49.235415 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.235328    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6868309a-8813-4420-bc7d-4a5b7a38eb46-plugins-config-volume" pod "6868309a-8813-4420-bc7d-4a5b7a38eb46" (UID: "6868309a-8813-4420-bc7d-4a5b7a38eb46"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:46:49.236174 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.236148    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6868309a-8813-4420-bc7d-4a5b7a38eb46-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:46:49.238906 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.238877    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6868309a-8813-4420-bc7d-4a5b7a38eb46-kube-api-access-n2ml2" pod "6868309a-8813-4420-bc7d-4a5b7a38eb46" (UID: "6868309a-8813-4420-bc7d-4a5b7a38eb46"). InnerVolumeSpecName "kube-api-access-n2ml2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:46:49.336807 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.336771    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/39007e3f-5368-4ae6-8d73-2e4d407cf76b-kube-api-access-ntgmx\" (UniqueName: \"kubernetes.io/projected/39007e3f-5368-4ae6-8d73-2e4d407cf76b-kube-api-access-ntgmx\") pod \"39007e3f-5368-4ae6-8d73-2e4d407cf76b\" (UID: \"39007e3f-5368-4ae6-8d73-2e4d407cf76b\") "
Oct 31 21:46:49.337024 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.336837    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/39007e3f-5368-4ae6-8d73-2e4d407cf76b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39007e3f-5368-4ae6-8d73-2e4d407cf76b-plugins-config-volume\") pod \"39007e3f-5368-4ae6-8d73-2e4d407cf76b\" (UID: \"39007e3f-5368-4ae6-8d73-2e4d407cf76b\") "
Oct 31 21:46:49.337338 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.337280    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39007e3f-5368-4ae6-8d73-2e4d407cf76b-plugins-config-volume" pod "39007e3f-5368-4ae6-8d73-2e4d407cf76b" (UID: "39007e3f-5368-4ae6-8d73-2e4d407cf76b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:46:49.337416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.337363    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2ml2\" (UniqueName: \"kubernetes.io/projected/6868309a-8813-4420-bc7d-4a5b7a38eb46-kube-api-access-n2ml2\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:46:49.338851 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.338819    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39007e3f-5368-4ae6-8d73-2e4d407cf76b-kube-api-access-ntgmx" pod "39007e3f-5368-4ae6-8d73-2e4d407cf76b" (UID: "39007e3f-5368-4ae6-8d73-2e4d407cf76b"). InnerVolumeSpecName "kube-api-access-ntgmx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:46:49.439441 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.439395    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ntgmx\" (UniqueName: \"kubernetes.io/projected/39007e3f-5368-4ae6-8d73-2e4d407cf76b-kube-api-access-ntgmx\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:46:49.439441 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.439436    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39007e3f-5368-4ae6-8d73-2e4d407cf76b-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:46:49.484418 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.484388    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:46:49.509498 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.509464    4460 generic.go:356] "Generic (PLEG): container finished" podID="39007e3f-5368-4ae6-8d73-2e4d407cf76b" containerID="f7baeb0b935c7f80a1c418bb81b2b79ff14069fae064004451712dc26bf6ba24" exitCode=0
Oct 31 21:46:49.509498 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.509499    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-80542b-777f94c5f9-p4p27"
Oct 31 21:46:49.509689 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.509497    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-80542b-777f94c5f9-p4p27" event={"ID":"39007e3f-5368-4ae6-8d73-2e4d407cf76b","Type":"ContainerDied","Data":"f7baeb0b935c7f80a1c418bb81b2b79ff14069fae064004451712dc26bf6ba24"}
Oct 31 21:46:49.509689 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.509617    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-80542b-777f94c5f9-p4p27" event={"ID":"39007e3f-5368-4ae6-8d73-2e4d407cf76b","Type":"ContainerDied","Data":"7e35c6209482de0adf4a7764ca1a109639b98ee1b6c887e2eb9ba84e3a12c794"}
Oct 31 21:46:49.509689 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.509639    4460 scope.go:122] "RemoveContainer" containerID="f7baeb0b935c7f80a1c418bb81b2b79ff14069fae064004451712dc26bf6ba24"
Oct 31 21:46:49.511389 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.511364    4460 generic.go:356] "Generic (PLEG): container finished" podID="78bb7115-5485-45ec-9b35-a76a75ba3eff" containerID="41d8d0da12535ca5380761cf68aa470090d3a575d8ae88a7af62410240b8ddd9" exitCode=0
Oct 31 21:46:49.511500 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.511427    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"
Oct 31 21:46:49.511500 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.511450    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns" event={"ID":"78bb7115-5485-45ec-9b35-a76a75ba3eff","Type":"ContainerDied","Data":"41d8d0da12535ca5380761cf68aa470090d3a575d8ae88a7af62410240b8ddd9"}
Oct 31 21:46:49.511610 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.511503    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns" event={"ID":"78bb7115-5485-45ec-9b35-a76a75ba3eff","Type":"ContainerDied","Data":"4d8f2f5a3890ebb7c2cccc4d7aaf633c2df7e7fea0ccf40b9d63e72917f3a3f5"}
Oct 31 21:46:49.513017 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.512993    4460 generic.go:356] "Generic (PLEG): container finished" podID="6868309a-8813-4420-bc7d-4a5b7a38eb46" containerID="86558919e547044d38698f268e7b27e614a6089b58638b8067e36b00dfa84620" exitCode=0
Oct 31 21:46:49.513121 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.513036    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-80542b-6c5b644b9-v8vqz"
Oct 31 21:46:49.513121 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.513050    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-80542b-6c5b644b9-v8vqz" event={"ID":"6868309a-8813-4420-bc7d-4a5b7a38eb46","Type":"ContainerDied","Data":"86558919e547044d38698f268e7b27e614a6089b58638b8067e36b00dfa84620"}
Oct 31 21:46:49.513121 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.513088    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-80542b-6c5b644b9-v8vqz" event={"ID":"6868309a-8813-4420-bc7d-4a5b7a38eb46","Type":"ContainerDied","Data":"479e392d63b536e77642dc78329de67c97abc0829840308a11b513ce9c09ff6a"}
Oct 31 21:46:49.521218 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.521199    4460 scope.go:122] "RemoveContainer" containerID="f7baeb0b935c7f80a1c418bb81b2b79ff14069fae064004451712dc26bf6ba24"
Oct 31 21:46:49.521604 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:46:49.521580    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7baeb0b935c7f80a1c418bb81b2b79ff14069fae064004451712dc26bf6ba24\": container with ID starting with f7baeb0b935c7f80a1c418bb81b2b79ff14069fae064004451712dc26bf6ba24 not found: ID does not exist" containerID="f7baeb0b935c7f80a1c418bb81b2b79ff14069fae064004451712dc26bf6ba24"
Oct 31 21:46:49.521716 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.521613    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7baeb0b935c7f80a1c418bb81b2b79ff14069fae064004451712dc26bf6ba24"} err="failed to get container status \"f7baeb0b935c7f80a1c418bb81b2b79ff14069fae064004451712dc26bf6ba24\": rpc error: code = NotFound desc = could not find container \"f7baeb0b935c7f80a1c418bb81b2b79ff14069fae064004451712dc26bf6ba24\": container with ID starting with f7baeb0b935c7f80a1c418bb81b2b79ff14069fae064004451712dc26bf6ba24 not found: ID does not exist"
Oct 31 21:46:49.521716 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.521634    4460 scope.go:122] "RemoveContainer" containerID="41d8d0da12535ca5380761cf68aa470090d3a575d8ae88a7af62410240b8ddd9"
Oct 31 21:46:49.531403 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.531385    4460 scope.go:122] "RemoveContainer" containerID="9b78df3386149b2efadc188f14b892e0d594f5cd5141821fc388f25309f65331"
Oct 31 21:46:49.542896 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.542871    4460 scope.go:122] "RemoveContainer" containerID="41d8d0da12535ca5380761cf68aa470090d3a575d8ae88a7af62410240b8ddd9"
Oct 31 21:46:49.543278 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:46:49.543255    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41d8d0da12535ca5380761cf68aa470090d3a575d8ae88a7af62410240b8ddd9\": container with ID starting with 41d8d0da12535ca5380761cf68aa470090d3a575d8ae88a7af62410240b8ddd9 not found: ID does not exist" containerID="41d8d0da12535ca5380761cf68aa470090d3a575d8ae88a7af62410240b8ddd9"
Oct 31 21:46:49.543381 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.543283    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41d8d0da12535ca5380761cf68aa470090d3a575d8ae88a7af62410240b8ddd9"} err="failed to get container status \"41d8d0da12535ca5380761cf68aa470090d3a575d8ae88a7af62410240b8ddd9\": rpc error: code = NotFound desc = could not find container \"41d8d0da12535ca5380761cf68aa470090d3a575d8ae88a7af62410240b8ddd9\": container with ID starting with 41d8d0da12535ca5380761cf68aa470090d3a575d8ae88a7af62410240b8ddd9 not found: ID does not exist"
Oct 31 21:46:49.543381 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.543300    4460 scope.go:122] "RemoveContainer" containerID="9b78df3386149b2efadc188f14b892e0d594f5cd5141821fc388f25309f65331"
Oct 31 21:46:49.543581 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:46:49.543561    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b78df3386149b2efadc188f14b892e0d594f5cd5141821fc388f25309f65331\": container with ID starting with 9b78df3386149b2efadc188f14b892e0d594f5cd5141821fc388f25309f65331 not found: ID does not exist" containerID="9b78df3386149b2efadc188f14b892e0d594f5cd5141821fc388f25309f65331"
Oct 31 21:46:49.543651 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.543589    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b78df3386149b2efadc188f14b892e0d594f5cd5141821fc388f25309f65331"} err="failed to get container status \"9b78df3386149b2efadc188f14b892e0d594f5cd5141821fc388f25309f65331\": rpc error: code = NotFound desc = could not find container \"9b78df3386149b2efadc188f14b892e0d594f5cd5141821fc388f25309f65331\": container with ID starting with 9b78df3386149b2efadc188f14b892e0d594f5cd5141821fc388f25309f65331 not found: ID does not exist"
Oct 31 21:46:49.543651 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.543608    4460 scope.go:122] "RemoveContainer" containerID="86558919e547044d38698f268e7b27e614a6089b58638b8067e36b00dfa84620"
Oct 31 21:46:49.554546 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.554525    4460 scope.go:122] "RemoveContainer" containerID="86558919e547044d38698f268e7b27e614a6089b58638b8067e36b00dfa84620"
Oct 31 21:46:49.554798 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:46:49.554770    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86558919e547044d38698f268e7b27e614a6089b58638b8067e36b00dfa84620\": container with ID starting with 86558919e547044d38698f268e7b27e614a6089b58638b8067e36b00dfa84620 not found: ID does not exist" containerID="86558919e547044d38698f268e7b27e614a6089b58638b8067e36b00dfa84620"
Oct 31 21:46:49.554868 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.554797    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86558919e547044d38698f268e7b27e614a6089b58638b8067e36b00dfa84620"} err="failed to get container status \"86558919e547044d38698f268e7b27e614a6089b58638b8067e36b00dfa84620\": rpc error: code = NotFound desc = could not find container \"86558919e547044d38698f268e7b27e614a6089b58638b8067e36b00dfa84620\": container with ID starting with 86558919e547044d38698f268e7b27e614a6089b58638b8067e36b00dfa84620 not found: ID does not exist"
Oct 31 21:46:49.591143 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.591109    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-80542b-777f94c5f9-p4p27"]
Oct 31 21:46:49.595299 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.595272    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-80542b-777f94c5f9-p4p27"]
Oct 31 21:46:49.631604 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.631573    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-80542b-6c5b644b9-v8vqz"]
Oct 31 21:46:49.637924 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.637893    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-80542b-6c5b644b9-v8vqz"]
Oct 31 21:46:49.642462 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.642443    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-istio-data\") pod \"78bb7115-5485-45ec-9b35-a76a75ba3eff\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") "
Oct 31 21:46:49.642537 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.642483    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-workload-certs\") pod \"78bb7115-5485-45ec-9b35-a76a75ba3eff\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") "
Oct 31 21:46:49.642537 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.642511    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-istio-envoy\") pod \"78bb7115-5485-45ec-9b35-a76a75ba3eff\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") "
Oct 31 21:46:49.642644 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.642558    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/78bb7115-5485-45ec-9b35-a76a75ba3eff-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78bb7115-5485-45ec-9b35-a76a75ba3eff-istio-podinfo\") pod \"78bb7115-5485-45ec-9b35-a76a75ba3eff\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") "
Oct 31 21:46:49.642644 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.642593    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-workload-socket\") pod \"78bb7115-5485-45ec-9b35-a76a75ba3eff\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") "
Oct 31 21:46:49.642756 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.642726    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-istio-data" pod "78bb7115-5485-45ec-9b35-a76a75ba3eff" (UID: "78bb7115-5485-45ec-9b35-a76a75ba3eff"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:46:49.642815 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.642762    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/78bb7115-5485-45ec-9b35-a76a75ba3eff-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78bb7115-5485-45ec-9b35-a76a75ba3eff-istiod-ca-cert\") pod \"78bb7115-5485-45ec-9b35-a76a75ba3eff\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") "
Oct 31 21:46:49.642868 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.642824    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/78bb7115-5485-45ec-9b35-a76a75ba3eff-kube-api-access-xtj9g\" (UniqueName: \"kubernetes.io/projected/78bb7115-5485-45ec-9b35-a76a75ba3eff-kube-api-access-xtj9g\") pod \"78bb7115-5485-45ec-9b35-a76a75ba3eff\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") "
Oct 31 21:46:49.642868 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.642831    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-workload-certs" pod "78bb7115-5485-45ec-9b35-a76a75ba3eff" (UID: "78bb7115-5485-45ec-9b35-a76a75ba3eff"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:46:49.643006 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.642909    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-credential-socket\") pod \"78bb7115-5485-45ec-9b35-a76a75ba3eff\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") "
Oct 31 21:46:49.643006 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.642983    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-workload-socket" pod "78bb7115-5485-45ec-9b35-a76a75ba3eff" (UID: "78bb7115-5485-45ec-9b35-a76a75ba3eff"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:46:49.643120 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.643018    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/78bb7115-5485-45ec-9b35-a76a75ba3eff-istio-token\" (UniqueName: \"kubernetes.io/projected/78bb7115-5485-45ec-9b35-a76a75ba3eff-istio-token\") pod \"78bb7115-5485-45ec-9b35-a76a75ba3eff\" (UID: \"78bb7115-5485-45ec-9b35-a76a75ba3eff\") "
Oct 31 21:46:49.643120 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.643109    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78bb7115-5485-45ec-9b35-a76a75ba3eff-istiod-ca-cert" pod "78bb7115-5485-45ec-9b35-a76a75ba3eff" (UID: "78bb7115-5485-45ec-9b35-a76a75ba3eff"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:46:49.643232 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.643164    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-credential-socket" pod "78bb7115-5485-45ec-9b35-a76a75ba3eff" (UID: "78bb7115-5485-45ec-9b35-a76a75ba3eff"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:46:49.643664 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.643641    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:46:49.643756 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.643671    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:46:49.643756 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.643687    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:46:49.643756 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.643699    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:46:49.643756 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.643712    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78bb7115-5485-45ec-9b35-a76a75ba3eff-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:46:49.644871 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.644846    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-istio-envoy" pod "78bb7115-5485-45ec-9b35-a76a75ba3eff" (UID: "78bb7115-5485-45ec-9b35-a76a75ba3eff"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:46:49.645002 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.644913    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/78bb7115-5485-45ec-9b35-a76a75ba3eff-istio-podinfo" pod "78bb7115-5485-45ec-9b35-a76a75ba3eff" (UID: "78bb7115-5485-45ec-9b35-a76a75ba3eff"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:46:49.645143 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.645125    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78bb7115-5485-45ec-9b35-a76a75ba3eff-kube-api-access-xtj9g" pod "78bb7115-5485-45ec-9b35-a76a75ba3eff" (UID: "78bb7115-5485-45ec-9b35-a76a75ba3eff"). InnerVolumeSpecName "kube-api-access-xtj9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:46:49.645183 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.645137    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78bb7115-5485-45ec-9b35-a76a75ba3eff-istio-token" pod "78bb7115-5485-45ec-9b35-a76a75ba3eff" (UID: "78bb7115-5485-45ec-9b35-a76a75ba3eff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:46:49.745295 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.745228    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78bb7115-5485-45ec-9b35-a76a75ba3eff-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:46:49.745295 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.745253    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78bb7115-5485-45ec-9b35-a76a75ba3eff-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:46:49.745295 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.745262    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78bb7115-5485-45ec-9b35-a76a75ba3eff-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:46:49.745295 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.745271    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtj9g\" (UniqueName: \"kubernetes.io/projected/78bb7115-5485-45ec-9b35-a76a75ba3eff-kube-api-access-xtj9g\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:46:49.862227 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.862197    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"]
Oct 31 21:46:49.865450 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:49.865425    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-a-448b01-openshift-default-6d6ff966b9-9xgns"]
Oct 31 21:46:50.228108 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:50.228075    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="39007e3f-5368-4ae6-8d73-2e4d407cf76b" path="/var/lib/kubelet/pods/39007e3f-5368-4ae6-8d73-2e4d407cf76b/volumes"
Oct 31 21:46:50.228677 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:50.228657    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6868309a-8813-4420-bc7d-4a5b7a38eb46" path="/var/lib/kubelet/pods/6868309a-8813-4420-bc7d-4a5b7a38eb46/volumes"
Oct 31 21:46:50.229333 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:50.229285    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="78bb7115-5485-45ec-9b35-a76a75ba3eff" path="/var/lib/kubelet/pods/78bb7115-5485-45ec-9b35-a76a75ba3eff/volumes"
Oct 31 21:46:50.542249 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:50.542220    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:51.459847 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:46:51.459807    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:46:52.328811 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:52.328779    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:54.030278 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:54.030243    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:55.506286 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:55.506244    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:55.617299 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:55.617262    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:55.725372 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:55.725341    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:56.065816 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:56.065782    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:56.177168 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:56.177127    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:56.321006 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:56.320844    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:56.688477 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:56.688382    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:58.133178 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:58.133143    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:46:59.851707 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:46:59.851668    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:01.743964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:01.743419    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:02.184111 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:02.184069    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:03.865194 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:03.865153    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:04.488487 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:04.488446    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:04.599413 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:04.599384    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:04.721973 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:04.720430    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:05.114415 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:05.114380    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:06.453005 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:06.452970    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:06.975350 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:06.975311    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"]
Oct 31 21:47:06.976004 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:06.975981    4460 cpu_manager.go:414] "containerMap: removing container" podUID="78bb7115-5485-45ec-9b35-a76a75ba3eff" containerName="istio-proxy"
Oct 31 21:47:06.976004 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:06.976003    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="78bb7115-5485-45ec-9b35-a76a75ba3eff" containerName="istio-proxy"
Oct 31 21:47:06.976185 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:06.976027    4460 cpu_manager.go:414] "containerMap: removing container" podUID="bad71d55-48bf-4120-affb-c627a9988e3e" containerName="payload-pre-processing"
Oct 31 21:47:06.976185 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:06.976034    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bad71d55-48bf-4120-affb-c627a9988e3e" containerName="payload-pre-processing"
Oct 31 21:47:06.976185 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:06.976051    4460 cpu_manager.go:414] "containerMap: removing container" podUID="39007e3f-5368-4ae6-8d73-2e4d407cf76b" containerName="payload-pre-processing"
Oct 31 21:47:06.976185 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:06.976058    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="39007e3f-5368-4ae6-8d73-2e4d407cf76b" containerName="payload-pre-processing"
Oct 31 21:47:06.976185 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:06.976099    4460 cpu_manager.go:414] "containerMap: removing container" podUID="6868309a-8813-4420-bc7d-4a5b7a38eb46" containerName="payload-processing"
Oct 31 21:47:06.976185 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:06.976106    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6868309a-8813-4420-bc7d-4a5b7a38eb46" containerName="payload-processing"
Oct 31 21:47:06.976185 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:06.976117    4460 cpu_manager.go:414] "containerMap: removing container" podUID="2113003b-165f-4533-aff5-85c75433f325" containerName="payload-processing"
Oct 31 21:47:06.976185 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:06.976123    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2113003b-165f-4533-aff5-85c75433f325" containerName="payload-processing"
Oct 31 21:47:06.976569 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:06.976284    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="bad71d55-48bf-4120-affb-c627a9988e3e" containerName="payload-pre-processing"
Oct 31 21:47:06.976569 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:06.976298    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="39007e3f-5368-4ae6-8d73-2e4d407cf76b" containerName="payload-pre-processing"
Oct 31 21:47:06.976569 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:06.976317    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="6868309a-8813-4420-bc7d-4a5b7a38eb46" containerName="payload-processing"
Oct 31 21:47:06.976569 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:06.976328    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="2113003b-165f-4533-aff5-85c75433f325" containerName="payload-processing"
Oct 31 21:47:06.976569 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:06.976343    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="78bb7115-5485-45ec-9b35-a76a75ba3eff" containerName="istio-proxy"
Oct 31 21:47:06.976569 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:06.976352    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="78bb7115-5485-45ec-9b35-a76a75ba3eff" containerName="istio-proxy"
Oct 31 21:47:06.982304 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:06.982281    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:47:07.029980 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.021643    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"]
Oct 31 21:47:07.029980 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.026697    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-workload-certs\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:47:07.029980 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.026776    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/082906e8-2fd4-4fdf-95a2-40a2e433982b-istio-podinfo\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:47:07.029980 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.026986    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-istio-envoy\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:47:07.029980 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.027373    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/082906e8-2fd4-4fdf-95a2-40a2e433982b-istio-token\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:47:07.029980 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.027531    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/082906e8-2fd4-4fdf-95a2-40a2e433982b-istiod-ca-cert\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:47:07.029980 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.027568    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-workload-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:47:07.029980 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.027688    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hwhr\" (UniqueName: \"kubernetes.io/projected/082906e8-2fd4-4fdf-95a2-40a2e433982b-kube-api-access-7hwhr\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:47:07.029980 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.027778    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-credential-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:47:07.029980 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.027844    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-istio-data\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:47:07.128735 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.128698    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-istio-data\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:47:07.128926 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.128811    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-workload-certs\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:47:07.128926 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.128871    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/082906e8-2fd4-4fdf-95a2-40a2e433982b-istio-podinfo\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:47:07.129100 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.128926    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-istio-envoy\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:47:07.129100 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.129056    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/082906e8-2fd4-4fdf-95a2-40a2e433982b-istio-token\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:47:07.129213 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.129121    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/082906e8-2fd4-4fdf-95a2-40a2e433982b-istiod-ca-cert\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:47:07.129213 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.129159    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-workload-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:47:07.129213 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.129168    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-istio-data\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:47:07.129213 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.129193    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hwhr\" (UniqueName: \"kubernetes.io/projected/082906e8-2fd4-4fdf-95a2-40a2e433982b-kube-api-access-7hwhr\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:47:07.129406 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.129282    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-credential-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:47:07.129406 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.129324    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-workload-certs\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:47:07.129550 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.129515    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-workload-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:47:07.129621 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.129590    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-credential-socket\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:47:07.129861 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.129834    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/082906e8-2fd4-4fdf-95a2-40a2e433982b-istiod-ca-cert\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:47:07.131241 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.131220    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-istio-envoy\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:47:07.131508 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.131489    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/082906e8-2fd4-4fdf-95a2-40a2e433982b-istio-podinfo\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:47:07.153899 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.153877    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/082906e8-2fd4-4fdf-95a2-40a2e433982b-istio-token\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:47:07.154304 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.154287    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hwhr\" (UniqueName: \"kubernetes.io/projected/082906e8-2fd4-4fdf-95a2-40a2e433982b-kube-api-access-7hwhr\") pod \"e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") " pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:47:07.298839 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.298807    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:47:07.625660 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.625630    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"]
Oct 31 21:47:07.626639 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:47:07.626617    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod082906e8_2fd4_4fdf_95a2_40a2e433982b.slice/crio-9c3701f0c5b31483bf19ca9a1d60adad5862aa0c993e74c13f8a539cce84ad5a WatchSource:0}: Error finding container 9c3701f0c5b31483bf19ca9a1d60adad5862aa0c993e74c13f8a539cce84ad5a: Status 404 returned error can't find the container with id 9c3701f0c5b31483bf19ca9a1d60adad5862aa0c993e74c13f8a539cce84ad5a
Oct 31 21:47:07.629227 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.629195    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:47:07.629286 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.629261    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:47:07.629321 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.629293    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:47:07.754291 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.754246    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m" event={"ID":"082906e8-2fd4-4fdf-95a2-40a2e433982b","Type":"ContainerStarted","Data":"3bc0bd7f8ba7588730430797c9d791203f359fa44dcc4e7d2f41d87b90fe5cb6"}
Oct 31 21:47:07.754466 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.754311    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m" event={"ID":"082906e8-2fd4-4fdf-95a2-40a2e433982b","Type":"ContainerStarted","Data":"9c3701f0c5b31483bf19ca9a1d60adad5862aa0c993e74c13f8a539cce84ad5a"}
Oct 31 21:47:07.805321 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:07.805264    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m" podStartSLOduration=1.8052491960000001 podStartE2EDuration="1.805249196s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:47:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:47:07.80252115 +0000 UTC m=+1867.718830580" watchObservedRunningTime="2026-10-31 21:47:07.805249196 +0000 UTC m=+1867.721558668"
Oct 31 21:47:08.244054 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:08.244013    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:08.313964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:08.311716    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:47:08.313964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:08.311861    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Oct 31 21:47:08.313964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:08.311910    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m" podUID="082906e8-2fd4-4fdf-95a2-40a2e433982b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Oct 31 21:47:09.299161 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:09.299125    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Oct 31 21:47:09.299552 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:09.299185    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m" podUID="082906e8-2fd4-4fdf-95a2-40a2e433982b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Oct 31 21:47:10.042965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:10.042504    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:10.299439 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:10.299344    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Oct 31 21:47:10.299439 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:10.299405    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m" podUID="082906e8-2fd4-4fdf-95a2-40a2e433982b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Oct 31 21:47:11.299327 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:11.299291    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Oct 31 21:47:11.299498 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:11.299346    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m" podUID="082906e8-2fd4-4fdf-95a2-40a2e433982b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Oct 31 21:47:11.766990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:11.766909    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:11.893527 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:11.893489    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:12.014968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:12.013718    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:12.182392 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:12.182306    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:12.316962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:12.312767    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Oct 31 21:47:12.316962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:12.312838    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m" podUID="082906e8-2fd4-4fdf-95a2-40a2e433982b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Oct 31 21:47:12.979804 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:12.979767    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:13.299194 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.299163    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Oct 31 21:47:13.299359 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.299222    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m" podUID="082906e8-2fd4-4fdf-95a2-40a2e433982b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Oct 31 21:47:13.402495 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.402464    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"]
Oct 31 21:47:13.403453 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.403429    4460 cpu_manager.go:414] "containerMap: removing container" podUID="78bb7115-5485-45ec-9b35-a76a75ba3eff" containerName="istio-proxy"
Oct 31 21:47:13.403453 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.403451    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="78bb7115-5485-45ec-9b35-a76a75ba3eff" containerName="istio-proxy"
Oct 31 21:47:13.407399 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.407379    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:47:13.455029 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.454994    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-b-ad121b-openshift-default-dockercfg-7s9fx\""
Oct 31 21:47:13.459377 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.459337    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"]
Oct 31 21:47:13.459827 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.459806    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:13.463035 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.463007    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-istio-data\") pod \"e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") " pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:47:13.463144 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.463059    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb421e61-04b4-4c83-b872-b21c58b24504-istio-podinfo\") pod \"e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") " pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:47:13.463204 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.463191    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb421e61-04b4-4c83-b872-b21c58b24504-istio-token\") pod \"e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") " pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:47:13.463261 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.463229    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-workload-certs\") pod \"e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") " pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:47:13.463324 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.463283    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-credential-socket\") pod \"e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") " pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:47:13.463378 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.463344    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-istio-envoy\") pod \"e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") " pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:47:13.463435 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.463413    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb421e61-04b4-4c83-b872-b21c58b24504-istiod-ca-cert\") pod \"e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") " pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:47:13.463529 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.463512    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-workload-socket\") pod \"e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") " pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:47:13.463590 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.463547    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgvm5\" (UniqueName: \"kubernetes.io/projected/eb421e61-04b4-4c83-b872-b21c58b24504-kube-api-access-fgvm5\") pod \"e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") " pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:47:13.600019 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.574152    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb421e61-04b4-4c83-b872-b21c58b24504-istio-podinfo\") pod \"e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") " pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:47:13.600454 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.600415    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb421e61-04b4-4c83-b872-b21c58b24504-istio-token\") pod \"e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") " pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:47:13.600653 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.600635    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-workload-certs\") pod \"e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") " pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:47:13.604134 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.604099    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-credential-socket\") pod \"e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") " pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:47:13.608074 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.608048    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-istio-envoy\") pod \"e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") " pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:47:13.608318 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.608297    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb421e61-04b4-4c83-b872-b21c58b24504-istiod-ca-cert\") pod \"e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") " pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:47:13.620962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.608525    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-workload-socket\") pod \"e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") " pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:47:13.621276 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.613329    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-workload-socket\") pod \"e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") " pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:47:13.621276 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.606820    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-workload-certs\") pod \"e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") " pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:47:13.621276 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.613934    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb421e61-04b4-4c83-b872-b21c58b24504-istiod-ca-cert\") pod \"e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") " pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:47:13.621276 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.605972    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-credential-socket\") pod \"e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") " pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:47:13.621276 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.618152    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-istio-envoy\") pod \"e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") " pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:47:13.621276 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.620412    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb421e61-04b4-4c83-b872-b21c58b24504-istio-podinfo\") pod \"e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") " pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:47:13.621898 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.621157    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgvm5\" (UniqueName: \"kubernetes.io/projected/eb421e61-04b4-4c83-b872-b21c58b24504-kube-api-access-fgvm5\") pod \"e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") " pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:47:13.622135 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.622117    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-istio-data\") pod \"e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") " pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:47:13.634974 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.622619    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-istio-data\") pod \"e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") " pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:47:13.666962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.653575    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb421e61-04b4-4c83-b872-b21c58b24504-istio-token\") pod \"e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") " pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:47:13.685964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.685664    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgvm5\" (UniqueName: \"kubernetes.io/projected/eb421e61-04b4-4c83-b872-b21c58b24504-kube-api-access-fgvm5\") pod \"e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") " pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:47:13.777261 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.777222    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:47:13.852022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.851653    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:13.975966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:13.975217    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:14.126593 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:14.126565    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"]
Oct 31 21:47:14.128257 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:47:14.128228    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb421e61_04b4_4c83_b872_b21c58b24504.slice/crio-b952a9681d77e2e1c027b112d7e2f4390a99d2a3b7e4801068b7375093210fc5 WatchSource:0}: Error finding container b952a9681d77e2e1c027b112d7e2f4390a99d2a3b7e4801068b7375093210fc5: Status 404 returned error can't find the container with id b952a9681d77e2e1c027b112d7e2f4390a99d2a3b7e4801068b7375093210fc5
Oct 31 21:47:14.131033 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:14.130999    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:47:14.131150 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:14.131067    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:47:14.131150 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:14.131108    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:47:14.299302 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:14.299264    4460 patch_prober.go:29] interesting pod/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Oct 31 21:47:14.299456 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:14.299326    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m" podUID="082906e8-2fd4-4fdf-95a2-40a2e433982b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Oct 31 21:47:14.809403 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:14.809365    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p" event={"ID":"eb421e61-04b4-4c83-b872-b21c58b24504","Type":"ContainerStarted","Data":"9a1f43ba4051c71ce2f9466c21e1925c9e6d7c500a7fae312b21df7af741577c"}
Oct 31 21:47:14.809806 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:14.809437    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p" event={"ID":"eb421e61-04b4-4c83-b872-b21c58b24504","Type":"ContainerStarted","Data":"b952a9681d77e2e1c027b112d7e2f4390a99d2a3b7e4801068b7375093210fc5"}
Oct 31 21:47:14.864743 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:14.864677    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p" podStartSLOduration=1.8646546960000001 podStartE2EDuration="1.864654696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:47:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:47:14.86230031 +0000 UTC m=+1874.778609743" watchObservedRunningTime="2026-10-31 21:47:14.864654696 +0000 UTC m=+1874.780964127"
Oct 31 21:47:15.373536 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:15.373499    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:47:15.373928 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:15.373904    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:47:15.374925 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:15.374905    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:47:15.777204 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:15.777167    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:15.778795 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:15.778770    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:47:15.784466 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:15.784445    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:47:15.879589 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:15.879538    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:47:15.881010 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:15.880676    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:47:17.296104 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:17.296072    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:17.410326 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:17.410288    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:17.579767 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:17.579680    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:18.997083 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:18.997050    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:19.110885 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:19.110845    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:19.422960 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:19.420216    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:21.121402 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:21.121368    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:21.452401 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:47:21.452312    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:47:22.273830 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:22.273793    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:22.394242 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:22.394211    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:22.920438 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:22.920396    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:23.012966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:22.994616    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:47:24.715928 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:24.715895    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:26.528190 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:26.528149    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:28.133211 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:28.133175    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:29.815355 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:29.815316    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:31.544410 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:31.544370    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:33.238257 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:33.238218    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:34.982205 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:34.982168    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:35.115966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:35.113620    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:35.291772 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:35.291744    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:36.195551 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:36.195510    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"]
Oct 31 21:47:36.203259 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:36.203217    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv" podUID="849e440a-7e18-4e25-90b3-bdc437076436" containerName="istio-proxy" containerID="cri-o://55f94ef8a50b7ba4e526e91f01ee45b433a397dc0739c946f1ad46bda2ee7458" gracePeriod=30
Oct 31 21:47:36.832826 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:36.832794    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:38.608181 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:38.608149    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:38.965545 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:38.965462    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:39.830981 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:39.830930    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:40.932062 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:40.932025    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:41.465659 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:41.465634    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:47:41.510773 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:41.510746    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-workload-socket\") pod \"849e440a-7e18-4e25-90b3-bdc437076436\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") "
Oct 31 21:47:41.510914 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:41.510782    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-credential-socket\") pod \"849e440a-7e18-4e25-90b3-bdc437076436\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") "
Oct 31 21:47:41.510914 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:41.510812    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-istio-envoy\") pod \"849e440a-7e18-4e25-90b3-bdc437076436\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") "
Oct 31 21:47:41.510914 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:41.510898    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/849e440a-7e18-4e25-90b3-bdc437076436-kube-api-access-dwkwj\" (UniqueName: \"kubernetes.io/projected/849e440a-7e18-4e25-90b3-bdc437076436-kube-api-access-dwkwj\") pod \"849e440a-7e18-4e25-90b3-bdc437076436\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") "
Oct 31 21:47:41.511358 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:41.510925    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-istio-data\") pod \"849e440a-7e18-4e25-90b3-bdc437076436\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") "
Oct 31 21:47:41.511358 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:41.510968    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/849e440a-7e18-4e25-90b3-bdc437076436-istio-token\" (UniqueName: \"kubernetes.io/projected/849e440a-7e18-4e25-90b3-bdc437076436-istio-token\") pod \"849e440a-7e18-4e25-90b3-bdc437076436\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") "
Oct 31 21:47:41.511358 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:41.511034    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-workload-socket" pod "849e440a-7e18-4e25-90b3-bdc437076436" (UID: "849e440a-7e18-4e25-90b3-bdc437076436"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:47:41.511358 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:41.511150    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-workload-certs\") pod \"849e440a-7e18-4e25-90b3-bdc437076436\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") "
Oct 31 21:47:41.511358 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:41.511191    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/849e440a-7e18-4e25-90b3-bdc437076436-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/849e440a-7e18-4e25-90b3-bdc437076436-istiod-ca-cert\") pod \"849e440a-7e18-4e25-90b3-bdc437076436\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") "
Oct 31 21:47:41.511358 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:41.511254    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/849e440a-7e18-4e25-90b3-bdc437076436-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/849e440a-7e18-4e25-90b3-bdc437076436-istio-podinfo\") pod \"849e440a-7e18-4e25-90b3-bdc437076436\" (UID: \"849e440a-7e18-4e25-90b3-bdc437076436\") "
Oct 31 21:47:41.511358 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:41.511250    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-istio-data" pod "849e440a-7e18-4e25-90b3-bdc437076436" (UID: "849e440a-7e18-4e25-90b3-bdc437076436"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:47:41.511720 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:41.511384    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-credential-socket" pod "849e440a-7e18-4e25-90b3-bdc437076436" (UID: "849e440a-7e18-4e25-90b3-bdc437076436"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:47:41.511720 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:41.511421    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-workload-certs" pod "849e440a-7e18-4e25-90b3-bdc437076436" (UID: "849e440a-7e18-4e25-90b3-bdc437076436"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:47:41.511720 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:41.511592    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/849e440a-7e18-4e25-90b3-bdc437076436-istiod-ca-cert" pod "849e440a-7e18-4e25-90b3-bdc437076436" (UID: "849e440a-7e18-4e25-90b3-bdc437076436"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:47:41.511971 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:41.511949    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:47:41.512049 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:41.511975    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:47:41.512049 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:41.511988    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/849e440a-7e18-4e25-90b3-bdc437076436-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:47:41.512049 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:41.512000    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:47:41.512049 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:41.512015    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:47:41.513408 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:41.513383    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/849e440a-7e18-4e25-90b3-bdc437076436-istio-podinfo" pod "849e440a-7e18-4e25-90b3-bdc437076436" (UID: "849e440a-7e18-4e25-90b3-bdc437076436"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:47:41.513522 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:41.513427    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-istio-envoy" pod "849e440a-7e18-4e25-90b3-bdc437076436" (UID: "849e440a-7e18-4e25-90b3-bdc437076436"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:47:41.513663 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:41.513642    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/849e440a-7e18-4e25-90b3-bdc437076436-kube-api-access-dwkwj" pod "849e440a-7e18-4e25-90b3-bdc437076436" (UID: "849e440a-7e18-4e25-90b3-bdc437076436"). InnerVolumeSpecName "kube-api-access-dwkwj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:47:41.513729 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:41.513675    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/849e440a-7e18-4e25-90b3-bdc437076436-istio-token" pod "849e440a-7e18-4e25-90b3-bdc437076436" (UID: "849e440a-7e18-4e25-90b3-bdc437076436"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:47:41.613370 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:41.613336    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwkwj\" (UniqueName: \"kubernetes.io/projected/849e440a-7e18-4e25-90b3-bdc437076436-kube-api-access-dwkwj\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:47:41.613370 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:41.613368    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/849e440a-7e18-4e25-90b3-bdc437076436-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:47:41.613370 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:41.613379    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/849e440a-7e18-4e25-90b3-bdc437076436-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:47:41.613603 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:41.613387    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/849e440a-7e18-4e25-90b3-bdc437076436-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:47:42.159765 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:42.159731    4460 generic.go:356] "Generic (PLEG): container finished" podID="849e440a-7e18-4e25-90b3-bdc437076436" containerID="55f94ef8a50b7ba4e526e91f01ee45b433a397dc0739c946f1ad46bda2ee7458" exitCode=0
Oct 31 21:47:42.160254 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:42.159773    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"
Oct 31 21:47:42.160254 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:42.159810    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv" event={"ID":"849e440a-7e18-4e25-90b3-bdc437076436","Type":"ContainerDied","Data":"55f94ef8a50b7ba4e526e91f01ee45b433a397dc0739c946f1ad46bda2ee7458"}
Oct 31 21:47:42.160254 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:42.159870    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv" event={"ID":"849e440a-7e18-4e25-90b3-bdc437076436","Type":"ContainerDied","Data":"c6ff6dfbb050f02129e97bb86c981a06165f1403281e7020590e25cf246c9c75"}
Oct 31 21:47:42.160254 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:42.159896    4460 scope.go:122] "RemoveContainer" containerID="55f94ef8a50b7ba4e526e91f01ee45b433a397dc0739c946f1ad46bda2ee7458"
Oct 31 21:47:42.172476 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:42.172456    4460 scope.go:122] "RemoveContainer" containerID="55f94ef8a50b7ba4e526e91f01ee45b433a397dc0739c946f1ad46bda2ee7458"
Oct 31 21:47:42.172783 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:47:42.172764    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55f94ef8a50b7ba4e526e91f01ee45b433a397dc0739c946f1ad46bda2ee7458\": container with ID starting with 55f94ef8a50b7ba4e526e91f01ee45b433a397dc0739c946f1ad46bda2ee7458 not found: ID does not exist" containerID="55f94ef8a50b7ba4e526e91f01ee45b433a397dc0739c946f1ad46bda2ee7458"
Oct 31 21:47:42.172843 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:42.172792    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55f94ef8a50b7ba4e526e91f01ee45b433a397dc0739c946f1ad46bda2ee7458"} err="failed to get container status \"55f94ef8a50b7ba4e526e91f01ee45b433a397dc0739c946f1ad46bda2ee7458\": rpc error: code = NotFound desc = could not find container \"55f94ef8a50b7ba4e526e91f01ee45b433a397dc0739c946f1ad46bda2ee7458\": container with ID starting with 55f94ef8a50b7ba4e526e91f01ee45b433a397dc0739c946f1ad46bda2ee7458 not found: ID does not exist"
Oct 31 21:47:42.232216 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:42.232191    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"]
Oct 31 21:47:42.245993 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:42.245970    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-l7qzv"]
Oct 31 21:47:42.670994 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:42.670949    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:44.149107 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:44.149060    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:44.226177 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:44.226147    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="849e440a-7e18-4e25-90b3-bdc437076436" path="/var/lib/kubelet/pods/849e440a-7e18-4e25-90b3-bdc437076436/volumes"
Oct 31 21:47:44.416477 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:44.416384    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:44.609358 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:44.609324    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:44.879529 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:44.879500    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:45.982760 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:45.982722    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:46.090760 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:46.090719    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:46.208689 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:46.208656    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:46.303982 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:46.302348    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"]
Oct 31 21:47:46.303982 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:46.302640    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk" podUID="928636a0-a99a-43da-b29f-6ad518f022e1" containerName="istio-proxy" containerID="cri-o://2e03ed96e4e8538b99401dfa62213aec023284ec897ea0ea8ed19c9a88cbd722" gracePeriod=30
Oct 31 21:47:46.353779 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:46.353741    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:47.061130 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:47.061091    4460 patch_prober.go:29] interesting pod/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 21:47:47.061586 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:47.061145    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk" podUID="928636a0-a99a-43da-b29f-6ad518f022e1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 21:47:47.399009 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:47.398907    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:48.054043 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:48.054008    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:48.168212 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:48.168165    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:48.449012 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:48.448897    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:48.895909 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:48.895875    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:49.063966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:49.063577    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:49.752951 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:49.752911    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:49.875401 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:49.875325    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:50.237391 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:50.237356    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:51.453506 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:47:51.453470    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:47:51.598342 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:51.598316    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:47:51.677472 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:51.677389    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-workload-socket\") pod \"928636a0-a99a-43da-b29f-6ad518f022e1\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") "
Oct 31 21:47:51.677472 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:51.677443    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/928636a0-a99a-43da-b29f-6ad518f022e1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/928636a0-a99a-43da-b29f-6ad518f022e1-istiod-ca-cert\") pod \"928636a0-a99a-43da-b29f-6ad518f022e1\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") "
Oct 31 21:47:51.677703 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:51.677476    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-istio-data\") pod \"928636a0-a99a-43da-b29f-6ad518f022e1\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") "
Oct 31 21:47:51.677703 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:51.677542    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/928636a0-a99a-43da-b29f-6ad518f022e1-kube-api-access-854dp\" (UniqueName: \"kubernetes.io/projected/928636a0-a99a-43da-b29f-6ad518f022e1-kube-api-access-854dp\") pod \"928636a0-a99a-43da-b29f-6ad518f022e1\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") "
Oct 31 21:47:51.677703 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:51.677601    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-workload-certs\") pod \"928636a0-a99a-43da-b29f-6ad518f022e1\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") "
Oct 31 21:47:51.677703 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:51.677679    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/928636a0-a99a-43da-b29f-6ad518f022e1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/928636a0-a99a-43da-b29f-6ad518f022e1-istio-podinfo\") pod \"928636a0-a99a-43da-b29f-6ad518f022e1\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") "
Oct 31 21:47:51.677703 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:51.677688    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-workload-socket" pod "928636a0-a99a-43da-b29f-6ad518f022e1" (UID: "928636a0-a99a-43da-b29f-6ad518f022e1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:47:51.678010 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:51.677757    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-istio-envoy\") pod \"928636a0-a99a-43da-b29f-6ad518f022e1\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") "
Oct 31 21:47:51.678010 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:51.677790    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-credential-socket\") pod \"928636a0-a99a-43da-b29f-6ad518f022e1\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") "
Oct 31 21:47:51.678010 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:51.677823    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/928636a0-a99a-43da-b29f-6ad518f022e1-istio-token\" (UniqueName: \"kubernetes.io/projected/928636a0-a99a-43da-b29f-6ad518f022e1-istio-token\") pod \"928636a0-a99a-43da-b29f-6ad518f022e1\" (UID: \"928636a0-a99a-43da-b29f-6ad518f022e1\") "
Oct 31 21:47:51.678010 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:51.677835    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-istio-data" pod "928636a0-a99a-43da-b29f-6ad518f022e1" (UID: "928636a0-a99a-43da-b29f-6ad518f022e1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:47:51.678010 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:51.677923    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-workload-certs" pod "928636a0-a99a-43da-b29f-6ad518f022e1" (UID: "928636a0-a99a-43da-b29f-6ad518f022e1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:47:51.678010 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:51.677977    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/928636a0-a99a-43da-b29f-6ad518f022e1-istiod-ca-cert" pod "928636a0-a99a-43da-b29f-6ad518f022e1" (UID: "928636a0-a99a-43da-b29f-6ad518f022e1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:47:51.678324 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:51.678152    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-credential-socket" pod "928636a0-a99a-43da-b29f-6ad518f022e1" (UID: "928636a0-a99a-43da-b29f-6ad518f022e1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:47:51.678602 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:51.678577    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:47:51.678712 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:51.678605    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/928636a0-a99a-43da-b29f-6ad518f022e1-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:47:51.678712 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:51.678620    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:47:51.678712 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:51.678633    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:47:51.678712 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:51.678645    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:47:51.680342 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:51.680314    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-istio-envoy" pod "928636a0-a99a-43da-b29f-6ad518f022e1" (UID: "928636a0-a99a-43da-b29f-6ad518f022e1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:47:51.680488 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:51.680467    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/928636a0-a99a-43da-b29f-6ad518f022e1-istio-token" pod "928636a0-a99a-43da-b29f-6ad518f022e1" (UID: "928636a0-a99a-43da-b29f-6ad518f022e1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:47:51.680551 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:51.680493    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/928636a0-a99a-43da-b29f-6ad518f022e1-istio-podinfo" pod "928636a0-a99a-43da-b29f-6ad518f022e1" (UID: "928636a0-a99a-43da-b29f-6ad518f022e1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:47:51.681931 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:51.681908    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/928636a0-a99a-43da-b29f-6ad518f022e1-kube-api-access-854dp" pod "928636a0-a99a-43da-b29f-6ad518f022e1" (UID: "928636a0-a99a-43da-b29f-6ad518f022e1"). InnerVolumeSpecName "kube-api-access-854dp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:47:51.779799 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:51.779768    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-854dp\" (UniqueName: \"kubernetes.io/projected/928636a0-a99a-43da-b29f-6ad518f022e1-kube-api-access-854dp\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:47:51.779799 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:51.779797    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/928636a0-a99a-43da-b29f-6ad518f022e1-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:47:51.780020 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:51.779810    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/928636a0-a99a-43da-b29f-6ad518f022e1-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:47:51.780020 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:51.779822    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/928636a0-a99a-43da-b29f-6ad518f022e1-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:47:52.122929 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:52.122887    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:52.246287 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:52.246251    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:52.283482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:52.283448    4460 generic.go:356] "Generic (PLEG): container finished" podID="928636a0-a99a-43da-b29f-6ad518f022e1" containerID="2e03ed96e4e8538b99401dfa62213aec023284ec897ea0ea8ed19c9a88cbd722" exitCode=0
Oct 31 21:47:52.283633 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:52.283496    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"
Oct 31 21:47:52.283633 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:52.283533    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk" event={"ID":"928636a0-a99a-43da-b29f-6ad518f022e1","Type":"ContainerDied","Data":"2e03ed96e4e8538b99401dfa62213aec023284ec897ea0ea8ed19c9a88cbd722"}
Oct 31 21:47:52.283633 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:52.283589    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk" event={"ID":"928636a0-a99a-43da-b29f-6ad518f022e1","Type":"ContainerDied","Data":"88b4f89f5938cbf3d989cdeab8cf62f3f8b4137623de8ad19deb9c56b46d4180"}
Oct 31 21:47:52.283633 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:52.283615    4460 scope.go:122] "RemoveContainer" containerID="2e03ed96e4e8538b99401dfa62213aec023284ec897ea0ea8ed19c9a88cbd722"
Oct 31 21:47:52.294761 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:52.294735    4460 scope.go:122] "RemoveContainer" containerID="b6242167b078c6ed8d290d52e301b250ee50f218aa5dc91fedb6634155510afc"
Oct 31 21:47:52.304156 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:52.304135    4460 scope.go:122] "RemoveContainer" containerID="2e03ed96e4e8538b99401dfa62213aec023284ec897ea0ea8ed19c9a88cbd722"
Oct 31 21:47:52.304437 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:47:52.304415    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e03ed96e4e8538b99401dfa62213aec023284ec897ea0ea8ed19c9a88cbd722\": container with ID starting with 2e03ed96e4e8538b99401dfa62213aec023284ec897ea0ea8ed19c9a88cbd722 not found: ID does not exist" containerID="2e03ed96e4e8538b99401dfa62213aec023284ec897ea0ea8ed19c9a88cbd722"
Oct 31 21:47:52.304503 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:52.304445    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e03ed96e4e8538b99401dfa62213aec023284ec897ea0ea8ed19c9a88cbd722"} err="failed to get container status \"2e03ed96e4e8538b99401dfa62213aec023284ec897ea0ea8ed19c9a88cbd722\": rpc error: code = NotFound desc = could not find container \"2e03ed96e4e8538b99401dfa62213aec023284ec897ea0ea8ed19c9a88cbd722\": container with ID starting with 2e03ed96e4e8538b99401dfa62213aec023284ec897ea0ea8ed19c9a88cbd722 not found: ID does not exist"
Oct 31 21:47:52.304503 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:52.304464    4460 scope.go:122] "RemoveContainer" containerID="b6242167b078c6ed8d290d52e301b250ee50f218aa5dc91fedb6634155510afc"
Oct 31 21:47:52.304712 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:47:52.304693    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6242167b078c6ed8d290d52e301b250ee50f218aa5dc91fedb6634155510afc\": container with ID starting with b6242167b078c6ed8d290d52e301b250ee50f218aa5dc91fedb6634155510afc not found: ID does not exist" containerID="b6242167b078c6ed8d290d52e301b250ee50f218aa5dc91fedb6634155510afc"
Oct 31 21:47:52.304763 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:52.304718    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6242167b078c6ed8d290d52e301b250ee50f218aa5dc91fedb6634155510afc"} err="failed to get container status \"b6242167b078c6ed8d290d52e301b250ee50f218aa5dc91fedb6634155510afc\": rpc error: code = NotFound desc = could not find container \"b6242167b078c6ed8d290d52e301b250ee50f218aa5dc91fedb6634155510afc\": container with ID starting with b6242167b078c6ed8d290d52e301b250ee50f218aa5dc91fedb6634155510afc not found: ID does not exist"
Oct 31 21:47:52.342803 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:52.342772    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"]
Oct 31 21:47:52.354952 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:52.354908    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-a-80542b-openshift-default-5f7fdb55bf-nhfkk"]
Oct 31 21:47:52.356255 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:52.356232    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:53.932431 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:53.932396    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:54.046381 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.046344    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:54.149512 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.149478    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0877f5-7b89c465c5-98ngt"]
Oct 31 21:47:54.150135 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.150113    4460 cpu_manager.go:414] "containerMap: removing container" podUID="928636a0-a99a-43da-b29f-6ad518f022e1" containerName="istio-proxy"
Oct 31 21:47:54.150135 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.150136    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="928636a0-a99a-43da-b29f-6ad518f022e1" containerName="istio-proxy"
Oct 31 21:47:54.150242 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.150166    4460 cpu_manager.go:414] "containerMap: removing container" podUID="849e440a-7e18-4e25-90b3-bdc437076436" containerName="istio-proxy"
Oct 31 21:47:54.150242 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.150172    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="849e440a-7e18-4e25-90b3-bdc437076436" containerName="istio-proxy"
Oct 31 21:47:54.150242 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.150181    4460 cpu_manager.go:414] "containerMap: removing container" podUID="928636a0-a99a-43da-b29f-6ad518f022e1" containerName="istio-proxy"
Oct 31 21:47:54.150242 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.150186    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="928636a0-a99a-43da-b29f-6ad518f022e1" containerName="istio-proxy"
Oct 31 21:47:54.150363 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.150273    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="849e440a-7e18-4e25-90b3-bdc437076436" containerName="istio-proxy"
Oct 31 21:47:54.150363 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.150283    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="928636a0-a99a-43da-b29f-6ad518f022e1" containerName="istio-proxy"
Oct 31 21:47:54.150363 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.150292    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="928636a0-a99a-43da-b29f-6ad518f022e1" containerName="istio-proxy"
Oct 31 21:47:54.155332 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.155309    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0877f5-7b89c465c5-98ngt"
Oct 31 21:47:54.163482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.163460    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:54.173867 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.173842    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0877f5-7b89c465c5-98ngt"]
Oct 31 21:47:54.182428 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.182395    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w0-0877f5\""
Oct 31 21:47:54.182582 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.182524    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w0-0877f5-dockercfg-mzff2\""
Oct 31 21:47:54.216582 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.216549    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-575n8\" (UniqueName: \"kubernetes.io/projected/bb34199a-aed8-4d77-97bf-2e738b867c96-kube-api-access-575n8\") pod \"payload-pre-processing-e2e-shared-a-w0-0877f5-7b89c465c5-98ngt\" (UID: \"bb34199a-aed8-4d77-97bf-2e738b867c96\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0877f5-7b89c465c5-98ngt"
Oct 31 21:47:54.216761 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.216733    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb34199a-aed8-4d77-97bf-2e738b867c96-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-0877f5-7b89c465c5-98ngt\" (UID: \"bb34199a-aed8-4d77-97bf-2e738b867c96\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0877f5-7b89c465c5-98ngt"
Oct 31 21:47:54.226627 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.226232    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="928636a0-a99a-43da-b29f-6ad518f022e1" path="/var/lib/kubelet/pods/928636a0-a99a-43da-b29f-6ad518f022e1/volumes"
Oct 31 21:47:54.352884 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.350535    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-575n8\" (UniqueName: \"kubernetes.io/projected/bb34199a-aed8-4d77-97bf-2e738b867c96-kube-api-access-575n8\") pod \"payload-pre-processing-e2e-shared-a-w0-0877f5-7b89c465c5-98ngt\" (UID: \"bb34199a-aed8-4d77-97bf-2e738b867c96\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0877f5-7b89c465c5-98ngt"
Oct 31 21:47:54.356773 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.356743    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb34199a-aed8-4d77-97bf-2e738b867c96-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-0877f5-7b89c465c5-98ngt\" (UID: \"bb34199a-aed8-4d77-97bf-2e738b867c96\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0877f5-7b89c465c5-98ngt"
Oct 31 21:47:54.358206 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.358179    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb34199a-aed8-4d77-97bf-2e738b867c96-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-0877f5-7b89c465c5-98ngt\" (UID: \"bb34199a-aed8-4d77-97bf-2e738b867c96\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0877f5-7b89c465c5-98ngt"
Oct 31 21:47:54.392428 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.392392    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-575n8\" (UniqueName: \"kubernetes.io/projected/bb34199a-aed8-4d77-97bf-2e738b867c96-kube-api-access-575n8\") pod \"payload-pre-processing-e2e-shared-a-w0-0877f5-7b89c465c5-98ngt\" (UID: \"bb34199a-aed8-4d77-97bf-2e738b867c96\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0877f5-7b89c465c5-98ngt"
Oct 31 21:47:54.403610 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.403570    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-0877f5-6c8d64c49f-8t5wq"]
Oct 31 21:47:54.409418 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.409387    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0877f5-6c8d64c49f-8t5wq"
Oct 31 21:47:54.429992 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.429933    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-0877f5-6c8d64c49f-8t5wq"]
Oct 31 21:47:54.471974 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.471910    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0877f5-7b89c465c5-98ngt"
Oct 31 21:47:54.474320 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.474285    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/249aadf1-c102-4ea3-b273-427c6df20f4f-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-0877f5-6c8d64c49f-8t5wq\" (UID: \"249aadf1-c102-4ea3-b273-427c6df20f4f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0877f5-6c8d64c49f-8t5wq"
Oct 31 21:47:54.475022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.474986    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp4pn\" (UniqueName: \"kubernetes.io/projected/249aadf1-c102-4ea3-b273-427c6df20f4f-kube-api-access-hp4pn\") pod \"payload-processing-e2e-shared-a-w0-0877f5-6c8d64c49f-8t5wq\" (UID: \"249aadf1-c102-4ea3-b273-427c6df20f4f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0877f5-6c8d64c49f-8t5wq"
Oct 31 21:47:54.576642 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.576604    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp4pn\" (UniqueName: \"kubernetes.io/projected/249aadf1-c102-4ea3-b273-427c6df20f4f-kube-api-access-hp4pn\") pod \"payload-processing-e2e-shared-a-w0-0877f5-6c8d64c49f-8t5wq\" (UID: \"249aadf1-c102-4ea3-b273-427c6df20f4f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0877f5-6c8d64c49f-8t5wq"
Oct 31 21:47:54.576803 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.576666    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/249aadf1-c102-4ea3-b273-427c6df20f4f-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-0877f5-6c8d64c49f-8t5wq\" (UID: \"249aadf1-c102-4ea3-b273-427c6df20f4f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0877f5-6c8d64c49f-8t5wq"
Oct 31 21:47:54.577537 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.577516    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/249aadf1-c102-4ea3-b273-427c6df20f4f-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-0877f5-6c8d64c49f-8t5wq\" (UID: \"249aadf1-c102-4ea3-b273-427c6df20f4f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0877f5-6c8d64c49f-8t5wq"
Oct 31 21:47:54.603096 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.603062    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp4pn\" (UniqueName: \"kubernetes.io/projected/249aadf1-c102-4ea3-b273-427c6df20f4f-kube-api-access-hp4pn\") pod \"payload-processing-e2e-shared-a-w0-0877f5-6c8d64c49f-8t5wq\" (UID: \"249aadf1-c102-4ea3-b273-427c6df20f4f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0877f5-6c8d64c49f-8t5wq"
Oct 31 21:47:54.772230 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.772202    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0877f5-6c8d64c49f-8t5wq"
Oct 31 21:47:54.785918 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:54.785894    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0877f5-7b89c465c5-98ngt"]
Oct 31 21:47:54.788007 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:47:54.787834    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb34199a_aed8_4d77_97bf_2e738b867c96.slice/crio-5d4bc7ccfb8d0c8033d9668cb435c45e7fdb32185d14241d99017b7c7dcc3a6a WatchSource:0}: Error finding container 5d4bc7ccfb8d0c8033d9668cb435c45e7fdb32185d14241d99017b7c7dcc3a6a: Status 404 returned error can't find the container with id 5d4bc7ccfb8d0c8033d9668cb435c45e7fdb32185d14241d99017b7c7dcc3a6a
Oct 31 21:47:55.079252 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:55.079224    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-0877f5-6c8d64c49f-8t5wq"]
Oct 31 21:47:55.079928 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:47:55.079903    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod249aadf1_c102_4ea3_b273_427c6df20f4f.slice/crio-6a8d6569f6c119d40fe5b7b3ad6dda53940be27ea21579feb995b7651c349da7 WatchSource:0}: Error finding container 6a8d6569f6c119d40fe5b7b3ad6dda53940be27ea21579feb995b7651c349da7: Status 404 returned error can't find the container with id 6a8d6569f6c119d40fe5b7b3ad6dda53940be27ea21579feb995b7651c349da7
Oct 31 21:47:55.339811 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:55.339712    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0877f5-6c8d64c49f-8t5wq" event={"ID":"249aadf1-c102-4ea3-b273-427c6df20f4f","Type":"ContainerStarted","Data":"27c7b1d849fbd7b3316e5886979232a3f4df89a2fc6dc0d8d777614b15037da1"}
Oct 31 21:47:55.339811 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:55.339772    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0877f5-6c8d64c49f-8t5wq" event={"ID":"249aadf1-c102-4ea3-b273-427c6df20f4f","Type":"ContainerStarted","Data":"6a8d6569f6c119d40fe5b7b3ad6dda53940be27ea21579feb995b7651c349da7"}
Oct 31 21:47:55.340036 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:55.339846    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0877f5-6c8d64c49f-8t5wq"
Oct 31 21:47:55.341430 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:55.341404    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0877f5-7b89c465c5-98ngt" event={"ID":"bb34199a-aed8-4d77-97bf-2e738b867c96","Type":"ContainerStarted","Data":"3344fc3644dbdf4625bd7578aa1a8ce69017ca24f9d58357b4a8248bbcf45da8"}
Oct 31 21:47:55.341618 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:55.341458    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0877f5-7b89c465c5-98ngt"
Oct 31 21:47:55.341618 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:55.341482    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0877f5-7b89c465c5-98ngt" event={"ID":"bb34199a-aed8-4d77-97bf-2e738b867c96","Type":"ContainerStarted","Data":"5d4bc7ccfb8d0c8033d9668cb435c45e7fdb32185d14241d99017b7c7dcc3a6a"}
Oct 31 21:47:55.385220 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:55.385176    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0877f5-6c8d64c49f-8t5wq" podStartSLOduration=1.385162868 podStartE2EDuration="1.385162868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:47:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:47:55.383516419 +0000 UTC m=+1915.299825843" watchObservedRunningTime="2026-10-31 21:47:55.385162868 +0000 UTC m=+1915.301472321"
Oct 31 21:47:55.435685 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:55.435641    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0877f5-7b89c465c5-98ngt" podStartSLOduration=1.435627431 podStartE2EDuration="1.435627431s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:47:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:47:55.431683081 +0000 UTC m=+1915.347992510" watchObservedRunningTime="2026-10-31 21:47:55.435627431 +0000 UTC m=+1915.351936859"
Oct 31 21:47:55.742724 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:55.742627    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:56.726962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:56.724829    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:47:57.409647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:57.409604    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:58.548571 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:58.548529    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-9c8b33-576597d459-mq54c"]
Oct 31 21:47:58.675171 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:58.675138    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:58.828800 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:58.807051    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:47:59.572646 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:47:59.572609    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:00.646374 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:00.646333    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:01.005738 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:01.005645    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:01.574848 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:01.574804    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:01.697234 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:01.697198    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:01.805964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:01.805869    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:02.222890 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:02.222850    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:03.009414 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:03.009382    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:03.548935 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:03.548898    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9c8b33-576597d459-mq54c" podUID="39674f94-37fe-47b6-b2a0-6449ada9d378" containerName="payload-pre-processing" containerID="cri-o://8e30f3b6fd07eab01a85bcbd64855f362f4f145ce6d1195c2b2dfe9cb0026f8f" gracePeriod=25
Oct 31 21:48:03.747427 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:03.745486    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:03.870070 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:03.870042    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9c8b33-576597d459-mq54c"
Oct 31 21:48:03.926249 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:03.926200    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/39674f94-37fe-47b6-b2a0-6449ada9d378-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39674f94-37fe-47b6-b2a0-6449ada9d378-plugins-config-volume\") pod \"39674f94-37fe-47b6-b2a0-6449ada9d378\" (UID: \"39674f94-37fe-47b6-b2a0-6449ada9d378\") "
Oct 31 21:48:03.926415 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:03.926255    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/39674f94-37fe-47b6-b2a0-6449ada9d378-kube-api-access-t74m4\" (UniqueName: \"kubernetes.io/projected/39674f94-37fe-47b6-b2a0-6449ada9d378-kube-api-access-t74m4\") pod \"39674f94-37fe-47b6-b2a0-6449ada9d378\" (UID: \"39674f94-37fe-47b6-b2a0-6449ada9d378\") "
Oct 31 21:48:03.927200 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:03.927172    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39674f94-37fe-47b6-b2a0-6449ada9d378-plugins-config-volume" pod "39674f94-37fe-47b6-b2a0-6449ada9d378" (UID: "39674f94-37fe-47b6-b2a0-6449ada9d378"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:48:03.945032 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:03.929547    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39674f94-37fe-47b6-b2a0-6449ada9d378-kube-api-access-t74m4" pod "39674f94-37fe-47b6-b2a0-6449ada9d378" (UID: "39674f94-37fe-47b6-b2a0-6449ada9d378"). InnerVolumeSpecName "kube-api-access-t74m4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:48:04.028608 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:04.028578    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39674f94-37fe-47b6-b2a0-6449ada9d378-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:48:04.028608 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:04.028604    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t74m4\" (UniqueName: \"kubernetes.io/projected/39674f94-37fe-47b6-b2a0-6449ada9d378-kube-api-access-t74m4\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:48:04.534040 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:04.534004    4460 generic.go:356] "Generic (PLEG): container finished" podID="39674f94-37fe-47b6-b2a0-6449ada9d378" containerID="8e30f3b6fd07eab01a85bcbd64855f362f4f145ce6d1195c2b2dfe9cb0026f8f" exitCode=0
Oct 31 21:48:04.534221 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:04.534066    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9c8b33-576597d459-mq54c"
Oct 31 21:48:04.534221 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:04.534091    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9c8b33-576597d459-mq54c" event={"ID":"39674f94-37fe-47b6-b2a0-6449ada9d378","Type":"ContainerDied","Data":"8e30f3b6fd07eab01a85bcbd64855f362f4f145ce6d1195c2b2dfe9cb0026f8f"}
Oct 31 21:48:04.534221 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:04.534173    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9c8b33-576597d459-mq54c" event={"ID":"39674f94-37fe-47b6-b2a0-6449ada9d378","Type":"ContainerDied","Data":"5575c966ddde6b3fccf8f1a6293fe8eacf2ffb98c9326b9aa42ad285d3b4e710"}
Oct 31 21:48:04.534221 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:04.534202    4460 scope.go:122] "RemoveContainer" containerID="8e30f3b6fd07eab01a85bcbd64855f362f4f145ce6d1195c2b2dfe9cb0026f8f"
Oct 31 21:48:04.544073 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:04.544050    4460 scope.go:122] "RemoveContainer" containerID="8e30f3b6fd07eab01a85bcbd64855f362f4f145ce6d1195c2b2dfe9cb0026f8f"
Oct 31 21:48:04.544416 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:48:04.544389    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e30f3b6fd07eab01a85bcbd64855f362f4f145ce6d1195c2b2dfe9cb0026f8f\": container with ID starting with 8e30f3b6fd07eab01a85bcbd64855f362f4f145ce6d1195c2b2dfe9cb0026f8f not found: ID does not exist" containerID="8e30f3b6fd07eab01a85bcbd64855f362f4f145ce6d1195c2b2dfe9cb0026f8f"
Oct 31 21:48:04.544501 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:04.544445    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e30f3b6fd07eab01a85bcbd64855f362f4f145ce6d1195c2b2dfe9cb0026f8f"} err="failed to get container status \"8e30f3b6fd07eab01a85bcbd64855f362f4f145ce6d1195c2b2dfe9cb0026f8f\": rpc error: code = NotFound desc = could not find container \"8e30f3b6fd07eab01a85bcbd64855f362f4f145ce6d1195c2b2dfe9cb0026f8f\": container with ID starting with 8e30f3b6fd07eab01a85bcbd64855f362f4f145ce6d1195c2b2dfe9cb0026f8f not found: ID does not exist"
Oct 31 21:48:04.590493 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:04.590460    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-9c8b33-576597d459-mq54c"]
Oct 31 21:48:04.598066 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:04.598042    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-9c8b33-576597d459-mq54c"]
Oct 31 21:48:05.545411 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:05.545379    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:06.225997 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:06.225966    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="39674f94-37fe-47b6-b2a0-6449ada9d378" path="/var/lib/kubelet/pods/39674f94-37fe-47b6-b2a0-6449ada9d378/volumes"
Oct 31 21:48:06.351373 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:06.351348    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0877f5-6c8d64c49f-8t5wq"
Oct 31 21:48:06.351553 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:06.351406    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0877f5-7b89c465c5-98ngt"
Oct 31 21:48:07.169150 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:07.169117    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:07.411964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:07.400607    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:07.515838 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:07.515807    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:08.845759 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:08.845718    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:08.952716 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:08.952687    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:09.155900 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.155815    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"]
Oct 31 21:48:09.156364 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.156331    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj" podUID="4e8f28aa-6c57-4882-92bf-b1c5f9450da5" containerName="istio-proxy" containerID="cri-o://07ba5963e1b9608123d9bc8f8c21d0e4b01d1a6131fa03b8e8617170e5d071c8" gracePeriod=30
Oct 31 21:48:09.194005 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.193930    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:09.734969 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.734902    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"]
Oct 31 21:48:09.735689 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.735663    4460 cpu_manager.go:414] "containerMap: removing container" podUID="39674f94-37fe-47b6-b2a0-6449ada9d378" containerName="payload-pre-processing"
Oct 31 21:48:09.735689 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.735684    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="39674f94-37fe-47b6-b2a0-6449ada9d378" containerName="payload-pre-processing"
Oct 31 21:48:09.735860 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.735784    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="39674f94-37fe-47b6-b2a0-6449ada9d378" containerName="payload-pre-processing"
Oct 31 21:48:09.741036 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.741011    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:48:09.779374 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.779342    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"]
Oct 31 21:48:09.825768 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.825736    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istiod-ca-cert\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:48:09.825928 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.825788    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-workload-certs\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:48:09.825928 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.825902    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-workload-socket\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:48:09.826065 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.825930    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istio-envoy\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:48:09.826132 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.826113    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istio-data\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:48:09.826171 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.826160    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istio-podinfo\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:48:09.826216 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.826179    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istio-token\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:48:09.826277 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.826264    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-credential-socket\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:48:09.826309 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.826288    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjmxp\" (UniqueName: \"kubernetes.io/projected/076f0db9-6f89-46d7-ad9c-9855f6dbf911-kube-api-access-gjmxp\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:48:09.927202 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.927159    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-credential-socket\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:48:09.927202 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.927207    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjmxp\" (UniqueName: \"kubernetes.io/projected/076f0db9-6f89-46d7-ad9c-9855f6dbf911-kube-api-access-gjmxp\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:48:09.927731 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.927275    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istiod-ca-cert\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:48:09.927731 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.927319    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-workload-certs\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:48:09.927731 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.927356    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-workload-socket\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:48:09.927731 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.927382    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istio-envoy\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:48:09.927731 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.927560    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istio-data\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:48:09.927731 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.927605    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-credential-socket\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:48:09.927731 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.927629    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istio-podinfo\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:48:09.927731 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.927661    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istio-token\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:48:09.928231 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.927742    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-workload-certs\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:48:09.928231 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.927824    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-workload-socket\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:48:09.928231 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.927915    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istio-data\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:48:09.928231 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.928181    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istiod-ca-cert\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:48:09.929651 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.929624    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istio-envoy\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:48:09.929892 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.929876    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istio-podinfo\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:48:09.952098 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.952074    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istio-token\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:48:09.952251 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:09.952233    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjmxp\" (UniqueName: \"kubernetes.io/projected/076f0db9-6f89-46d7-ad9c-9855f6dbf911-kube-api-access-gjmxp\") pod \"e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") " pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:48:10.056460 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.056431    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:48:10.374762 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.374730    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"]
Oct 31 21:48:10.376072 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:48:10.376044    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod076f0db9_6f89_46d7_ad9c_9855f6dbf911.slice/crio-a953f891cf4ec0a2ed798fe944369f137c033553b01a297ba11a020694e6831f WatchSource:0}: Error finding container a953f891cf4ec0a2ed798fe944369f137c033553b01a297ba11a020694e6831f: Status 404 returned error can't find the container with id a953f891cf4ec0a2ed798fe944369f137c033553b01a297ba11a020694e6831f
Oct 31 21:48:10.378665 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.378631    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:48:10.378777 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.378706    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:48:10.378777 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.378751    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:48:10.401709 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.401681    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:10.588161 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.588121    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b" event={"ID":"076f0db9-6f89-46d7-ad9c-9855f6dbf911","Type":"ContainerStarted","Data":"eabf64413d846d3573428a026d107ae35cb064c113f7673be654fe35fc12a26c"}
Oct 31 21:48:10.588357 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.588178    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b" event={"ID":"076f0db9-6f89-46d7-ad9c-9855f6dbf911","Type":"ContainerStarted","Data":"a953f891cf4ec0a2ed798fe944369f137c033553b01a297ba11a020694e6831f"}
Oct 31 21:48:10.640401 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.640302    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b" podStartSLOduration=1.64028828 podStartE2EDuration="1.64028828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:48:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:48:10.637348442 +0000 UTC m=+1930.553657884" watchObservedRunningTime="2026-10-31 21:48:10.64028828 +0000 UTC m=+1930.556597708"
Oct 31 21:48:10.654648 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.654612    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:10.793492 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.793461    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"]
Oct 31 21:48:10.798827 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.798800    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:48:10.819008 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.818975    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-6b5dab-openshift-default-dockercfg-qnq2v\""
Oct 31 21:48:10.821810 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.821782    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"]
Oct 31 21:48:10.840042 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.840016    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-istio-data\") pod \"e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") " pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:48:10.840042 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.840051    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-workload-socket\") pod \"e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") " pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:48:10.840242 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.840081    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/581aed43-b880-434d-a7ca-3d0c78e54c38-istiod-ca-cert\") pod \"e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") " pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:48:10.840242 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.840219    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-istio-envoy\") pod \"e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") " pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:48:10.840358 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.840338    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/581aed43-b880-434d-a7ca-3d0c78e54c38-istio-podinfo\") pod \"e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") " pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:48:10.840393 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.840378    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/581aed43-b880-434d-a7ca-3d0c78e54c38-istio-token\") pod \"e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") " pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:48:10.840428 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.840399    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-credential-socket\") pod \"e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") " pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:48:10.840544 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.840526    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-workload-certs\") pod \"e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") " pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:48:10.840657 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.840616    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bd6dl\" (UniqueName: \"kubernetes.io/projected/581aed43-b880-434d-a7ca-3d0c78e54c38-kube-api-access-bd6dl\") pod \"e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") " pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:48:10.851302 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.851180    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:10.957011 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.949998    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bd6dl\" (UniqueName: \"kubernetes.io/projected/581aed43-b880-434d-a7ca-3d0c78e54c38-kube-api-access-bd6dl\") pod \"e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") " pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:48:10.957011 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.950191    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-istio-data\") pod \"e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") " pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:48:10.957011 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.950225    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-workload-socket\") pod \"e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") " pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:48:10.957011 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.950265    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/581aed43-b880-434d-a7ca-3d0c78e54c38-istiod-ca-cert\") pod \"e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") " pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:48:10.957011 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.950303    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-istio-envoy\") pod \"e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") " pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:48:10.957011 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.950380    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/581aed43-b880-434d-a7ca-3d0c78e54c38-istio-podinfo\") pod \"e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") " pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:48:10.957011 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.950415    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/581aed43-b880-434d-a7ca-3d0c78e54c38-istio-token\") pod \"e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") " pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:48:10.957011 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.950442    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-credential-socket\") pod \"e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") " pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:48:10.957011 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.950533    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-workload-certs\") pod \"e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") " pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:48:10.957011 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.951042    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-workload-certs\") pod \"e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") " pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:48:10.957011 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.951641    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-istio-data\") pod \"e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") " pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:48:10.957011 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.951880    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-workload-socket\") pod \"e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") " pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:48:10.957011 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.952509    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/581aed43-b880-434d-a7ca-3d0c78e54c38-istiod-ca-cert\") pod \"e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") " pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:48:10.985973 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:10.983000    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-credential-socket\") pod \"e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") " pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:48:11.002756 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:11.002718    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:11.004123 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:11.004090    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-istio-envoy\") pod \"e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") " pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:48:11.004752 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:11.004718    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bd6dl\" (UniqueName: \"kubernetes.io/projected/581aed43-b880-434d-a7ca-3d0c78e54c38-kube-api-access-bd6dl\") pod \"e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") " pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:48:11.029999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:11.022353    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/581aed43-b880-434d-a7ca-3d0c78e54c38-istio-podinfo\") pod \"e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") " pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:48:11.029999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:11.025327    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/581aed43-b880-434d-a7ca-3d0c78e54c38-istio-token\") pod \"e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") " pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:48:11.069964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:11.062801    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:48:11.073488 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:11.071063    4460 patch_prober.go:29] interesting pod/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused" start-of-body=
Oct 31 21:48:11.073488 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:11.071124    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b" podUID="076f0db9-6f89-46d7-ad9c-9855f6dbf911" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused"
Oct 31 21:48:11.123965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:11.121237    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:48:11.528685 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:11.528646    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"]
Oct 31 21:48:11.547144 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:11.547103    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:48:11.547821 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:11.547791    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:48:11.547988 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:11.547868    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:48:11.605142 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:11.605107    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw" event={"ID":"581aed43-b880-434d-a7ca-3d0c78e54c38","Type":"ContainerStarted","Data":"a6d6e20b3adb71576f3dc2a25a0de81c44b6847f0d48e69204666c2f12b6a0bd"}
Oct 31 21:48:12.057729 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:12.057691    4460 patch_prober.go:29] interesting pod/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused" start-of-body=
Oct 31 21:48:12.058123 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:12.057750    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b" podUID="076f0db9-6f89-46d7-ad9c-9855f6dbf911" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused"
Oct 31 21:48:12.472777 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:12.472666    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:12.583452 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:12.583405    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:12.612518 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:12.612482    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw" event={"ID":"581aed43-b880-434d-a7ca-3d0c78e54c38","Type":"ContainerStarted","Data":"2eb8df3c9a4d36a0a64881d84caeb3cde2c7883b7f41698a162f7c0fa2083a54"}
Oct 31 21:48:12.669960 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:12.669411    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw" podStartSLOduration=2.669392754 podStartE2EDuration="2.669392754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:48:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:48:12.669222956 +0000 UTC m=+1932.585532386" watchObservedRunningTime="2026-10-31 21:48:12.669392754 +0000 UTC m=+1932.585702186"
Oct 31 21:48:12.725022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:12.720585    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:13.057571 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:13.057539    4460 patch_prober.go:29] interesting pod/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused" start-of-body=
Oct 31 21:48:13.057753 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:13.057592    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b" podUID="076f0db9-6f89-46d7-ad9c-9855f6dbf911" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused"
Oct 31 21:48:13.137958 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:13.137494    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:48:13.149163 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:13.149136    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:48:13.675963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:13.665135    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:48:13.678057 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:13.677046    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:48:14.061513 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:14.061475    4460 patch_prober.go:29] interesting pod/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused" start-of-body=
Oct 31 21:48:14.062081 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:14.061554    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b" podUID="076f0db9-6f89-46d7-ad9c-9855f6dbf911" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused"
Oct 31 21:48:14.768930 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:14.764393    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:14.871975 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:14.860652    4460 generic.go:356] "Generic (PLEG): container finished" podID="4e8f28aa-6c57-4882-92bf-b1c5f9450da5" containerID="07ba5963e1b9608123d9bc8f8c21d0e4b01d1a6131fa03b8e8617170e5d071c8" exitCode=0
Oct 31 21:48:14.871975 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:14.861845    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj" event={"ID":"4e8f28aa-6c57-4882-92bf-b1c5f9450da5","Type":"ContainerDied","Data":"07ba5963e1b9608123d9bc8f8c21d0e4b01d1a6131fa03b8e8617170e5d071c8"}
Oct 31 21:48:14.871975 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:14.861914    4460 scope.go:122] "RemoveContainer" containerID="a0dd4795966350e151c1d7302266d13aa94c85669f67dcfb49fe143dab683cd8"
Oct 31 21:48:14.883534 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:14.883514    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:48:15.005815 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.005782    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istio-podinfo\") pod \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") "
Oct 31 21:48:15.006050 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.005856    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istio-token\" (UniqueName: \"kubernetes.io/projected/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istio-token\") pod \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") "
Oct 31 21:48:15.006050 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.005879    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istiod-ca-cert\") pod \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") "
Oct 31 21:48:15.006050 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.005995    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-workload-certs\") pod \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") "
Oct 31 21:48:15.006255 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.006073    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istio-data\") pod \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") "
Oct 31 21:48:15.006255 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.006157    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istio-envoy\") pod \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") "
Oct 31 21:48:15.006255 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.006191    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-workload-socket\") pod \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") "
Oct 31 21:48:15.006255 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.006239    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-kube-api-access-xxff6\" (UniqueName: \"kubernetes.io/projected/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-kube-api-access-xxff6\") pod \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") "
Oct 31 21:48:15.006478 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.006335    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-credential-socket\") pod \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\" (UID: \"4e8f28aa-6c57-4882-92bf-b1c5f9450da5\") "
Oct 31 21:48:15.006478 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.006338    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istiod-ca-cert" pod "4e8f28aa-6c57-4882-92bf-b1c5f9450da5" (UID: "4e8f28aa-6c57-4882-92bf-b1c5f9450da5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:48:15.006609 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.006493    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istio-data" pod "4e8f28aa-6c57-4882-92bf-b1c5f9450da5" (UID: "4e8f28aa-6c57-4882-92bf-b1c5f9450da5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:48:15.006665 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.006622    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-workload-socket" pod "4e8f28aa-6c57-4882-92bf-b1c5f9450da5" (UID: "4e8f28aa-6c57-4882-92bf-b1c5f9450da5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:48:15.007165 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.007138    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-workload-certs" pod "4e8f28aa-6c57-4882-92bf-b1c5f9450da5" (UID: "4e8f28aa-6c57-4882-92bf-b1c5f9450da5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:48:15.007374 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.007327    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-credential-socket" pod "4e8f28aa-6c57-4882-92bf-b1c5f9450da5" (UID: "4e8f28aa-6c57-4882-92bf-b1c5f9450da5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:48:15.007873 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.007849    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:48:15.007996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.007879    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:48:15.007996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.007893    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:48:15.008774 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.008552    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istio-token" pod "4e8f28aa-6c57-4882-92bf-b1c5f9450da5" (UID: "4e8f28aa-6c57-4882-92bf-b1c5f9450da5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:48:15.009428 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.009379    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istio-envoy" pod "4e8f28aa-6c57-4882-92bf-b1c5f9450da5" (UID: "4e8f28aa-6c57-4882-92bf-b1c5f9450da5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:48:15.009768 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.009738    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istio-podinfo" pod "4e8f28aa-6c57-4882-92bf-b1c5f9450da5" (UID: "4e8f28aa-6c57-4882-92bf-b1c5f9450da5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:48:15.009876 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.009849    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-kube-api-access-xxff6" pod "4e8f28aa-6c57-4882-92bf-b1c5f9450da5" (UID: "4e8f28aa-6c57-4882-92bf-b1c5f9450da5"). InnerVolumeSpecName "kube-api-access-xxff6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:48:15.060269 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.059766    4460 patch_prober.go:29] interesting pod/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused" start-of-body=
Oct 31 21:48:15.060269 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.059844    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b" podUID="076f0db9-6f89-46d7-ad9c-9855f6dbf911" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused"
Oct 31 21:48:15.109404 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.109367    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:48:15.109404 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.109400    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:48:15.109404 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.109409    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:48:15.109838 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.109417    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxff6\" (UniqueName: \"kubernetes.io/projected/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-kube-api-access-xxff6\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:48:15.109838 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.109426    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:48:15.109838 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.109434    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e8f28aa-6c57-4882-92bf-b1c5f9450da5-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:48:15.145645 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.145570    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:15.269143 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.269104    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:15.381747 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.381659    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:15.492752 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.492715    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:15.878176 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.878139    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"
Oct 31 21:48:15.878507 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.878476    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj" event={"ID":"4e8f28aa-6c57-4882-92bf-b1c5f9450da5","Type":"ContainerDied","Data":"e2408be4c49a9d91f30e29f4b08305d1213bf26f6b4cf66a4d02cd26c7ec8aa7"}
Oct 31 21:48:15.878591 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.878546    4460 scope.go:122] "RemoveContainer" containerID="07ba5963e1b9608123d9bc8f8c21d0e4b01d1a6131fa03b8e8617170e5d071c8"
Oct 31 21:48:15.988194 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.988161    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"]
Oct 31 21:48:15.996343 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:15.996311    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-b-9c8b33-openshift-default-798b6b79b9-2ftxj"]
Oct 31 21:48:16.056966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:16.056808    4460 patch_prober.go:29] interesting pod/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused" start-of-body=
Oct 31 21:48:16.056966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:16.056882    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b" podUID="076f0db9-6f89-46d7-ad9c-9855f6dbf911" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused"
Oct 31 21:48:16.284715 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:16.284684    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4e8f28aa-6c57-4882-92bf-b1c5f9450da5" path="/var/lib/kubelet/pods/4e8f28aa-6c57-4882-92bf-b1c5f9450da5/volumes"
Oct 31 21:48:16.908273 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:16.908237    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:17.020322 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.020287    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:17.057334 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.057294    4460 patch_prober.go:29] interesting pod/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused" start-of-body=
Oct 31 21:48:17.057469 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.057373    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b" podUID="076f0db9-6f89-46d7-ad9c-9855f6dbf911" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.143:15021/healthz/ready\": dial tcp 10.133.0.143:15021: connect: connection refused"
Oct 31 21:48:17.087208 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.087176    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-6b5dab-75bfb5c8df-nh5kr"]
Oct 31 21:48:17.088196 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.088175    4460 cpu_manager.go:414] "containerMap: removing container" podUID="4e8f28aa-6c57-4882-92bf-b1c5f9450da5" containerName="istio-proxy"
Oct 31 21:48:17.088281 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.088198    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4e8f28aa-6c57-4882-92bf-b1c5f9450da5" containerName="istio-proxy"
Oct 31 21:48:17.088281 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.088230    4460 cpu_manager.go:414] "containerMap: removing container" podUID="4e8f28aa-6c57-4882-92bf-b1c5f9450da5" containerName="istio-proxy"
Oct 31 21:48:17.088281 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.088238    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4e8f28aa-6c57-4882-92bf-b1c5f9450da5" containerName="istio-proxy"
Oct 31 21:48:17.088422 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.088409    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="4e8f28aa-6c57-4882-92bf-b1c5f9450da5" containerName="istio-proxy"
Oct 31 21:48:17.088457 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.088431    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="4e8f28aa-6c57-4882-92bf-b1c5f9450da5" containerName="istio-proxy"
Oct 31 21:48:17.093957 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.093914    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-6b5dab-75bfb5c8df-nh5kr"
Oct 31 21:48:17.118074 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.118051    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-6b5dab-75bfb5c8df-nh5kr"]
Oct 31 21:48:17.126642 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.126620    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-6b5dab\""
Oct 31 21:48:17.143392 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.143357    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:17.150346 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.150316    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-6b5dab-59c4c79ff9-n89dv"]
Oct 31 21:48:17.155521 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.155503    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-6b5dab-59c4c79ff9-n89dv"
Oct 31 21:48:17.172602 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.172577    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-6b5dab-59c4c79ff9-n89dv"]
Oct 31 21:48:17.173126 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.173102    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-6b5dab-dockercfg-4h757\""
Oct 31 21:48:17.180346 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.180320    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afe24009-66f9-4972-9e31-709ec4fa2acb-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-6b5dab-75bfb5c8df-nh5kr\" (UID: \"afe24009-66f9-4972-9e31-709ec4fa2acb\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-6b5dab-75bfb5c8df-nh5kr"
Oct 31 21:48:17.180500 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.180480    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpgpt\" (UniqueName: \"kubernetes.io/projected/15a6d325-3256-4d12-b698-fffafe8c39f0-kube-api-access-kpgpt\") pod \"payload-pre-processing-e2e-shared-a-w2-6b5dab-59c4c79ff9-n89dv\" (UID: \"15a6d325-3256-4d12-b698-fffafe8c39f0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-6b5dab-59c4c79ff9-n89dv"
Oct 31 21:48:17.180736 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.180708    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hqxg\" (UniqueName: \"kubernetes.io/projected/afe24009-66f9-4972-9e31-709ec4fa2acb-kube-api-access-9hqxg\") pod \"payload-processing-e2e-shared-a-w2-6b5dab-75bfb5c8df-nh5kr\" (UID: \"afe24009-66f9-4972-9e31-709ec4fa2acb\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-6b5dab-75bfb5c8df-nh5kr"
Oct 31 21:48:17.180845 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.180757    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15a6d325-3256-4d12-b698-fffafe8c39f0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-6b5dab-59c4c79ff9-n89dv\" (UID: \"15a6d325-3256-4d12-b698-fffafe8c39f0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-6b5dab-59c4c79ff9-n89dv"
Oct 31 21:48:17.283036 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.282985    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hqxg\" (UniqueName: \"kubernetes.io/projected/afe24009-66f9-4972-9e31-709ec4fa2acb-kube-api-access-9hqxg\") pod \"payload-processing-e2e-shared-a-w2-6b5dab-75bfb5c8df-nh5kr\" (UID: \"afe24009-66f9-4972-9e31-709ec4fa2acb\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-6b5dab-75bfb5c8df-nh5kr"
Oct 31 21:48:17.283229 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.283051    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15a6d325-3256-4d12-b698-fffafe8c39f0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-6b5dab-59c4c79ff9-n89dv\" (UID: \"15a6d325-3256-4d12-b698-fffafe8c39f0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-6b5dab-59c4c79ff9-n89dv"
Oct 31 21:48:17.283229 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.283173    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afe24009-66f9-4972-9e31-709ec4fa2acb-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-6b5dab-75bfb5c8df-nh5kr\" (UID: \"afe24009-66f9-4972-9e31-709ec4fa2acb\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-6b5dab-75bfb5c8df-nh5kr"
Oct 31 21:48:17.283359 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.283297    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpgpt\" (UniqueName: \"kubernetes.io/projected/15a6d325-3256-4d12-b698-fffafe8c39f0-kube-api-access-kpgpt\") pod \"payload-pre-processing-e2e-shared-a-w2-6b5dab-59c4c79ff9-n89dv\" (UID: \"15a6d325-3256-4d12-b698-fffafe8c39f0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-6b5dab-59c4c79ff9-n89dv"
Oct 31 21:48:17.284389 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.284361    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afe24009-66f9-4972-9e31-709ec4fa2acb-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-6b5dab-75bfb5c8df-nh5kr\" (UID: \"afe24009-66f9-4972-9e31-709ec4fa2acb\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-6b5dab-75bfb5c8df-nh5kr"
Oct 31 21:48:17.284876 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.284859    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15a6d325-3256-4d12-b698-fffafe8c39f0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-6b5dab-59c4c79ff9-n89dv\" (UID: \"15a6d325-3256-4d12-b698-fffafe8c39f0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-6b5dab-59c4c79ff9-n89dv"
Oct 31 21:48:17.290908 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.290883    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:17.310912 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.310887    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpgpt\" (UniqueName: \"kubernetes.io/projected/15a6d325-3256-4d12-b698-fffafe8c39f0-kube-api-access-kpgpt\") pod \"payload-pre-processing-e2e-shared-a-w2-6b5dab-59c4c79ff9-n89dv\" (UID: \"15a6d325-3256-4d12-b698-fffafe8c39f0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-6b5dab-59c4c79ff9-n89dv"
Oct 31 21:48:17.311199 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.311179    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hqxg\" (UniqueName: \"kubernetes.io/projected/afe24009-66f9-4972-9e31-709ec4fa2acb-kube-api-access-9hqxg\") pod \"payload-processing-e2e-shared-a-w2-6b5dab-75bfb5c8df-nh5kr\" (UID: \"afe24009-66f9-4972-9e31-709ec4fa2acb\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-6b5dab-75bfb5c8df-nh5kr"
Oct 31 21:48:17.411876 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.411847    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-6b5dab-75bfb5c8df-nh5kr"
Oct 31 21:48:17.476974 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.472254    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-6b5dab-59c4c79ff9-n89dv"
Oct 31 21:48:17.882966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.882923    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-6b5dab-75bfb5c8df-nh5kr"]
Oct 31 21:48:17.883559 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:48:17.883535    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafe24009_66f9_4972_9e31_709ec4fa2acb.slice/crio-c05a5b0ed67c75c2e9e06e7b0785dd5a20c9a49918c3817d61b0e7a0e962e26b WatchSource:0}: Error finding container c05a5b0ed67c75c2e9e06e7b0785dd5a20c9a49918c3817d61b0e7a0e962e26b: Status 404 returned error can't find the container with id c05a5b0ed67c75c2e9e06e7b0785dd5a20c9a49918c3817d61b0e7a0e962e26b
Oct 31 21:48:17.894385 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:17.894349    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-6b5dab-75bfb5c8df-nh5kr" event={"ID":"afe24009-66f9-4972-9e31-709ec4fa2acb","Type":"ContainerStarted","Data":"c05a5b0ed67c75c2e9e06e7b0785dd5a20c9a49918c3817d61b0e7a0e962e26b"}
Oct 31 21:48:18.023341 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:18.023310    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-6b5dab-59c4c79ff9-n89dv"]
Oct 31 21:48:18.024800 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:48:18.024772    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15a6d325_3256_4d12_b698_fffafe8c39f0.slice/crio-f112a00ddae28d1af7694ab081d91a25fae836136d44b35ef653b13e7e5c13f4 WatchSource:0}: Error finding container f112a00ddae28d1af7694ab081d91a25fae836136d44b35ef653b13e7e5c13f4: Status 404 returned error can't find the container with id f112a00ddae28d1af7694ab081d91a25fae836136d44b35ef653b13e7e5c13f4
Oct 31 21:48:18.263289 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:18.263257    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:48:18.263794 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:18.263774    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:48:18.264722 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:18.264701    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:48:18.887312 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:18.887280    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:18.902951 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:18.902898    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-6b5dab-59c4c79ff9-n89dv" event={"ID":"15a6d325-3256-4d12-b698-fffafe8c39f0","Type":"ContainerStarted","Data":"c826d9add18ef144496b83592c85332bf74d18da80d5972625ebb8ed5493ba9a"}
Oct 31 21:48:18.903121 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:18.903085    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-6b5dab-59c4c79ff9-n89dv"
Oct 31 21:48:18.903189 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:18.903120    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-6b5dab-59c4c79ff9-n89dv" event={"ID":"15a6d325-3256-4d12-b698-fffafe8c39f0","Type":"ContainerStarted","Data":"f112a00ddae28d1af7694ab081d91a25fae836136d44b35ef653b13e7e5c13f4"}
Oct 31 21:48:18.904530 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:18.904496    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-6b5dab-75bfb5c8df-nh5kr" event={"ID":"afe24009-66f9-4972-9e31-709ec4fa2acb","Type":"ContainerStarted","Data":"ff31202536e63724b16d8793d4bee396ff047b09e5ba5cc72974944c7baa7c23"}
Oct 31 21:48:18.904718 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:18.904700    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-6b5dab-75bfb5c8df-nh5kr"
Oct 31 21:48:18.969638 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:18.952404    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-6b5dab-59c4c79ff9-n89dv" podStartSLOduration=1.952384441 podStartE2EDuration="1.952384441s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:48:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:48:18.95043813 +0000 UTC m=+1938.866747558" watchObservedRunningTime="2026-10-31 21:48:18.952384441 +0000 UTC m=+1938.868693871"
Oct 31 21:48:19.008038 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:18.998193    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-6b5dab-75bfb5c8df-nh5kr" podStartSLOduration=1.9981736749999999 podStartE2EDuration="1.998173675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:48:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:48:18.997180108 +0000 UTC m=+1938.913489539" watchObservedRunningTime="2026-10-31 21:48:18.998173675 +0000 UTC m=+1938.914483106"
Oct 31 21:48:20.613593 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:20.613558    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:21.452895 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:48:21.452859    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:48:22.313447 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:22.313413    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:23.785270 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:23.785235    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:24.325016 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:24.324923    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:24.435321 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:24.435284    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:24.545597 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:24.545557    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:24.951772 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:24.951738    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:26.416234 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:26.416201    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:28.046740 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:28.046699    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:29.724246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:29.724209    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:29.913838 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:29.913809    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-6b5dab-59c4c79ff9-n89dv"
Oct 31 21:48:29.914052 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:29.913896    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-6b5dab-75bfb5c8df-nh5kr"
Oct 31 21:48:31.540750 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:31.540707    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:32.183405 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:32.183364    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:32.290170 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:32.290133    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:32.542438 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:32.542402    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:33.515579 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:33.515542    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:33.691090 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:33.691058    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:34.150644 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:34.150603    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:34.602884 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:34.602849    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:34.929955 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:34.929852    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:35.401075 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:35.401040    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:35.511361 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:35.511328    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:35.645968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:35.642030    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:37.427522 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:37.427488    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:37.868746 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:37.868713    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:38.044216 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:38.044175    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:48:38.293093 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:38.293064    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:38.605762 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:38.605594    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:38.728863 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:38.728826    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:38.844734 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:38.844693    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:38.964619 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:38.964526    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:39.635381 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:39.635345    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:40.319350 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:40.319316    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:40.440159 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:40.440111    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:40.751014 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:40.750017    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-fb32d4-b4fd9fb87-l6tzn"]
Oct 31 21:48:40.803682 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:40.803645    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-fb32d4-b4fd9fb87-l6tzn"]
Oct 31 21:48:40.804068 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:40.804050    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fb32d4-b4fd9fb87-l6tzn"
Oct 31 21:48:40.820917 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:40.820893    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w1-fb32d4\""
Oct 31 21:48:40.834973 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:40.834923    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w1-fb32d4-dockercfg-kshq4\""
Oct 31 21:48:40.899973 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:40.899930    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/982141b1-3b70-4767-9442-264f8920c0b5-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-fb32d4-b4fd9fb87-l6tzn\" (UID: \"982141b1-3b70-4767-9442-264f8920c0b5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fb32d4-b4fd9fb87-l6tzn"
Oct 31 21:48:40.900116 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:40.900095    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7gjt\" (UniqueName: \"kubernetes.io/projected/982141b1-3b70-4767-9442-264f8920c0b5-kube-api-access-s7gjt\") pod \"payload-processing-e2e-shared-b-w1-fb32d4-b4fd9fb87-l6tzn\" (UID: \"982141b1-3b70-4767-9442-264f8920c0b5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fb32d4-b4fd9fb87-l6tzn"
Oct 31 21:48:41.001007 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:41.000979    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/982141b1-3b70-4767-9442-264f8920c0b5-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-fb32d4-b4fd9fb87-l6tzn\" (UID: \"982141b1-3b70-4767-9442-264f8920c0b5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fb32d4-b4fd9fb87-l6tzn"
Oct 31 21:48:41.001172 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:41.001070    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7gjt\" (UniqueName: \"kubernetes.io/projected/982141b1-3b70-4767-9442-264f8920c0b5-kube-api-access-s7gjt\") pod \"payload-processing-e2e-shared-b-w1-fb32d4-b4fd9fb87-l6tzn\" (UID: \"982141b1-3b70-4767-9442-264f8920c0b5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fb32d4-b4fd9fb87-l6tzn"
Oct 31 21:48:41.001710 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:41.001691    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/982141b1-3b70-4767-9442-264f8920c0b5-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-fb32d4-b4fd9fb87-l6tzn\" (UID: \"982141b1-3b70-4767-9442-264f8920c0b5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fb32d4-b4fd9fb87-l6tzn"
Oct 31 21:48:41.026253 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:41.026227    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7gjt\" (UniqueName: \"kubernetes.io/projected/982141b1-3b70-4767-9442-264f8920c0b5-kube-api-access-s7gjt\") pod \"payload-processing-e2e-shared-b-w1-fb32d4-b4fd9fb87-l6tzn\" (UID: \"982141b1-3b70-4767-9442-264f8920c0b5\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fb32d4-b4fd9fb87-l6tzn"
Oct 31 21:48:41.129549 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:41.129524    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fb32d4-b4fd9fb87-l6tzn"
Oct 31 21:48:41.445788 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:41.445757    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-fb32d4-b4fd9fb87-l6tzn"]
Oct 31 21:48:41.446269 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:48:41.446242    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod982141b1_3b70_4767_9442_264f8920c0b5.slice/crio-f88dd3eeb4aad082e9ac1fb5d8ebcd0197aa17d8c0c714bf069bdf20aedb843e WatchSource:0}: Error finding container f88dd3eeb4aad082e9ac1fb5d8ebcd0197aa17d8c0c714bf069bdf20aedb843e: Status 404 returned error can't find the container with id f88dd3eeb4aad082e9ac1fb5d8ebcd0197aa17d8c0c714bf069bdf20aedb843e
Oct 31 21:48:41.524239 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:41.524204    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:42.149825 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:42.149790    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fb32d4-b4fd9fb87-l6tzn" event={"ID":"982141b1-3b70-4767-9442-264f8920c0b5","Type":"ContainerStarted","Data":"8124a6b000290299b000b239acbbf112d08f06b4383471f166faafd1f2350a8b"}
Oct 31 21:48:42.150283 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:42.149845    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fb32d4-b4fd9fb87-l6tzn" event={"ID":"982141b1-3b70-4767-9442-264f8920c0b5","Type":"ContainerStarted","Data":"f88dd3eeb4aad082e9ac1fb5d8ebcd0197aa17d8c0c714bf069bdf20aedb843e"}
Oct 31 21:48:42.150283 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:42.149907    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fb32d4-b4fd9fb87-l6tzn"
Oct 31 21:48:42.196124 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:42.196079    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fb32d4-b4fd9fb87-l6tzn" podStartSLOduration=2.196065445 podStartE2EDuration="2.196065445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:48:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:48:42.193597007 +0000 UTC m=+1962.109906439" watchObservedRunningTime="2026-10-31 21:48:42.196065445 +0000 UTC m=+1962.112374873"
Oct 31 21:48:42.854052 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:42.854015    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:42.975196 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:42.975158    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:43.451215 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:43.451184    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:43.867628 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:43.867593    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:43.996970 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:43.996903    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:45.145962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:45.145808    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:45.254703 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:45.254662    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:45.374915 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:45.374879    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:47.117522 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:47.117486    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:47.507643 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:47.507552    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:48.838573 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:48.838535    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:50.531867 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:50.531835    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:51.454713 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:48:51.454670    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:48:52.231047 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:52.231013    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:53.164897 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:53.164870    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fb32d4-b4fd9fb87-l6tzn"
Oct 31 21:48:54.028062 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:54.027980    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:54.351952 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:54.351915    4460 generic.go:356] "Generic (PLEG): container finished" podID="eb421e61-04b4-4c83-b872-b21c58b24504" containerID="9a1f43ba4051c71ce2f9466c21e1925c9e6d7c500a7fae312b21df7af741577c" exitCode=0
Oct 31 21:48:54.352128 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:54.351987    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p" event={"ID":"eb421e61-04b4-4c83-b872-b21c58b24504","Type":"ContainerDied","Data":"9a1f43ba4051c71ce2f9466c21e1925c9e6d7c500a7fae312b21df7af741577c"}
Oct 31 21:48:54.352365 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:54.352343    4460 scope.go:122] "RemoveContainer" containerID="9a1f43ba4051c71ce2f9466c21e1925c9e6d7c500a7fae312b21df7af741577c"
Oct 31 21:48:54.353856 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:54.353825    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:48:54.354008 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:54.353892    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:48:54.354008 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:54.353926    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:48:55.376477 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:55.376439    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p" event={"ID":"eb421e61-04b4-4c83-b872-b21c58b24504","Type":"ContainerStarted","Data":"8f2f684c35a872d5d46d15ff905db809864f8ca44a3f2695bb9975a5480a940f"}
Oct 31 21:48:55.751650 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:55.751548    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:55.778969 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:55.778920    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:48:55.779266 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:55.779119    4460 patch_prober.go:29] interesting pod/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused" start-of-body=
Oct 31 21:48:55.779458 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:55.779425    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p" podUID="eb421e61-04b4-4c83-b872-b21c58b24504" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused"
Oct 31 21:48:56.778555 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:56.778511    4460 patch_prober.go:29] interesting pod/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused" start-of-body=
Oct 31 21:48:56.778994 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:56.778577    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p" podUID="eb421e61-04b4-4c83-b872-b21c58b24504" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused"
Oct 31 21:48:57.366963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:57.366579    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:57.779959 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:57.779236    4460 patch_prober.go:29] interesting pod/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused" start-of-body=
Oct 31 21:48:57.779959 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:57.779287    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p" podUID="eb421e61-04b4-4c83-b872-b21c58b24504" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused"
Oct 31 21:48:58.779192 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:58.779149    4460 patch_prober.go:29] interesting pod/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused" start-of-body=
Oct 31 21:48:58.779366 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:58.779208    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p" podUID="eb421e61-04b4-4c83-b872-b21c58b24504" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused"
Oct 31 21:48:59.183313 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:59.183215    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:59.292501 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:59.292448    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:59.422760 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:59.422716    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:48:59.778870 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:59.778830    4460 patch_prober.go:29] interesting pod/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused" start-of-body=
Oct 31 21:48:59.779052 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:48:59.778889    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p" podUID="eb421e61-04b4-4c83-b872-b21c58b24504" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.140:15021/healthz/ready\": dial tcp 10.133.0.140:15021: connect: connection refused"
Oct 31 21:49:00.784797 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:00.784767    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:49:00.785309 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:00.785289    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:49:00.786227 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:00.786194    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:49:00.824418 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:00.824382    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:01.187384 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:01.187280    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:01.302626 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:01.302590    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:01.722967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:01.719804    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"]
Oct 31 21:49:01.725371 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:01.725344    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:49:01.745539 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:01.745501    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"]
Oct 31 21:49:01.765455 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:01.765416    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:01.783382 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:01.783343    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-44405d-openshift-default-dockercfg-vjr7p\""
Oct 31 21:49:01.850961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:01.845908    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd21505e-c4de-43eb-a5da-5c69c7c90785-istio-podinfo\") pod \"e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") " pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:49:01.850961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:01.845998    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-workload-socket\") pod \"e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") " pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:49:01.850961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:01.846061    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-workload-certs\") pod \"e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") " pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:49:01.850961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:01.846094    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-credential-socket\") pod \"e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") " pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:49:01.850961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:01.846125    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pf6jz\" (UniqueName: \"kubernetes.io/projected/dd21505e-c4de-43eb-a5da-5c69c7c90785-kube-api-access-pf6jz\") pod \"e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") " pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:49:01.850961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:01.846343    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd21505e-c4de-43eb-a5da-5c69c7c90785-istiod-ca-cert\") pod \"e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") " pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:49:01.850961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:01.846381    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-istio-envoy\") pod \"e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") " pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:49:01.850961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:01.846444    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd21505e-c4de-43eb-a5da-5c69c7c90785-istio-token\") pod \"e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") " pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:49:01.850961 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:01.846515    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-istio-data\") pod \"e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") " pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:49:01.955963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:01.951839    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-workload-certs\") pod \"e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") " pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:49:01.955963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:01.951911    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-credential-socket\") pod \"e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") " pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:49:01.955963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:01.951968    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pf6jz\" (UniqueName: \"kubernetes.io/projected/dd21505e-c4de-43eb-a5da-5c69c7c90785-kube-api-access-pf6jz\") pod \"e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") " pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:49:01.955963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:01.952201    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd21505e-c4de-43eb-a5da-5c69c7c90785-istiod-ca-cert\") pod \"e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") " pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:49:01.955963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:01.952236    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-istio-envoy\") pod \"e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") " pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:49:01.955963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:01.952294    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd21505e-c4de-43eb-a5da-5c69c7c90785-istio-token\") pod \"e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") " pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:49:01.955963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:01.952357    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-istio-data\") pod \"e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") " pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:49:01.955963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:01.952427    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd21505e-c4de-43eb-a5da-5c69c7c90785-istio-podinfo\") pod \"e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") " pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:49:01.955963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:01.952461    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-workload-socket\") pod \"e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") " pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:49:01.955963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:01.953249    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-workload-socket\") pod \"e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") " pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:49:01.955963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:01.953489    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-workload-certs\") pod \"e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") " pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:49:01.955963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:01.953714    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-credential-socket\") pod \"e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") " pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:49:01.955963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:01.954827    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd21505e-c4de-43eb-a5da-5c69c7c90785-istiod-ca-cert\") pod \"e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") " pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:49:01.957075 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:01.957052    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-istio-data\") pod \"e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") " pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:49:01.987596 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:01.987484    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd21505e-c4de-43eb-a5da-5c69c7c90785-istio-podinfo\") pod \"e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") " pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:49:02.006490 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:02.006453    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd21505e-c4de-43eb-a5da-5c69c7c90785-istio-token\") pod \"e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") " pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:49:02.022446 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:02.022410    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-istio-envoy\") pod \"e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") " pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:49:02.024863 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:02.024841    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pf6jz\" (UniqueName: \"kubernetes.io/projected/dd21505e-c4de-43eb-a5da-5c69c7c90785-kube-api-access-pf6jz\") pod \"e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") " pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:49:02.047055 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:02.047028    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:02.047815 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:02.047797    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:49:02.443697 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:02.443669    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"]
Oct 31 21:49:02.444624 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:49:02.444596    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd21505e_c4de_43eb_a5da_5c69c7c90785.slice/crio-dc33b3bed7cd3a0c88e83e8e9a27e509d56ab9974121f1f1f8ff7e61b0e581c2 WatchSource:0}: Error finding container dc33b3bed7cd3a0c88e83e8e9a27e509d56ab9974121f1f1f8ff7e61b0e581c2: Status 404 returned error can't find the container with id dc33b3bed7cd3a0c88e83e8e9a27e509d56ab9974121f1f1f8ff7e61b0e581c2
Oct 31 21:49:02.447416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:02.447380    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:49:02.447518 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:02.447464    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:49:02.447582 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:02.447526    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:49:02.472603 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:02.472560    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp" event={"ID":"dd21505e-c4de-43eb-a5da-5c69c7c90785","Type":"ContainerStarted","Data":"dc33b3bed7cd3a0c88e83e8e9a27e509d56ab9974121f1f1f8ff7e61b0e581c2"}
Oct 31 21:49:02.995971 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:02.995920    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:03.480192 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:03.480157    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp" event={"ID":"dd21505e-c4de-43eb-a5da-5c69c7c90785","Type":"ContainerStarted","Data":"0012406ddf0805880548028d005622769699f85c53a707bed83322e99b8f90e2"}
Oct 31 21:49:03.543670 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:03.543618    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp" podStartSLOduration=2.543604643 podStartE2EDuration="2.543604643s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:49:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:49:03.541564303 +0000 UTC m=+1983.457873733" watchObservedRunningTime="2026-10-31 21:49:03.543604643 +0000 UTC m=+1983.459914072"
Oct 31 21:49:04.049035 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:04.049009    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:49:04.055020 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:04.054998    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:49:04.416357 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:04.416308    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:04.499283 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:04.499251    4460 generic.go:356] "Generic (PLEG): container finished" podID="581aed43-b880-434d-a7ca-3d0c78e54c38" containerID="2eb8df3c9a4d36a0a64881d84caeb3cde2c7883b7f41698a162f7c0fa2083a54" exitCode=0
Oct 31 21:49:04.499447 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:04.499316    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw" event={"ID":"581aed43-b880-434d-a7ca-3d0c78e54c38","Type":"ContainerDied","Data":"2eb8df3c9a4d36a0a64881d84caeb3cde2c7883b7f41698a162f7c0fa2083a54"}
Oct 31 21:49:04.499578 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:04.499550    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:49:04.499723 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:04.499712    4460 scope.go:122] "RemoveContainer" containerID="2eb8df3c9a4d36a0a64881d84caeb3cde2c7883b7f41698a162f7c0fa2083a54"
Oct 31 21:49:04.500526 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:04.500509    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:49:04.501525 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:04.501495    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:49:04.501593 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:04.501575    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:49:04.501630 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:04.501618    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:49:05.287231 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:05.287199    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:05.396475 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:05.396436    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:05.509696 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:05.509607    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:05.513739 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:05.513708    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw" event={"ID":"581aed43-b880-434d-a7ca-3d0c78e54c38","Type":"ContainerStarted","Data":"03436e419060bbbf2273b87c3eddb77ea8b46003dc00bd65b0e678525bcb7ad7"}
Oct 31 21:49:05.636864 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:05.636784    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:06.010192 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:06.010158    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:06.119807 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:06.119770    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:06.124962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:06.123041    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:49:06.124962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:06.123153    4460 patch_prober.go:29] interesting pod/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused" start-of-body=
Oct 31 21:49:06.124962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:06.123211    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw" podUID="581aed43-b880-434d-a7ca-3d0c78e54c38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused"
Oct 31 21:49:07.125963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:07.125391    4460 patch_prober.go:29] interesting pod/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused" start-of-body=
Oct 31 21:49:07.125963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:07.125467    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw" podUID="581aed43-b880-434d-a7ca-3d0c78e54c38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused"
Oct 31 21:49:07.391665 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:07.391578    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:08.128366 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:08.128329    4460 patch_prober.go:29] interesting pod/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused" start-of-body=
Oct 31 21:49:08.129026 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:08.128993    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw" podUID="581aed43-b880-434d-a7ca-3d0c78e54c38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused"
Oct 31 21:49:08.544963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:08.544288    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:08.655325 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:08.655273    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:09.146243 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:09.146197    4460 patch_prober.go:29] interesting pod/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused" start-of-body=
Oct 31 21:49:09.146678 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:09.146257    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw" podUID="581aed43-b880-434d-a7ca-3d0c78e54c38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused"
Oct 31 21:49:09.247170 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:09.246773    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:10.136997 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:10.134083    4460 patch_prober.go:29] interesting pod/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused" start-of-body=
Oct 31 21:49:10.136997 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:10.134155    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw" podUID="581aed43-b880-434d-a7ca-3d0c78e54c38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.144:15021/healthz/ready\": dial tcp 10.133.0.144:15021: connect: connection refused"
Oct 31 21:49:11.037162 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:11.037110    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:11.140264 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:11.140230    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:49:11.140740 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:11.140720    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:49:11.141415 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:11.141397    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 21:49:12.730741 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:12.730701    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:12.854248 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:12.854213    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:13.044219 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:13.044182    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"]
Oct 31 21:49:13.044420 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:13.044398    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p" podUID="eb421e61-04b4-4c83-b872-b21c58b24504" containerName="istio-proxy" containerID="cri-o://8f2f684c35a872d5d46d15ff905db809864f8ca44a3f2695bb9975a5480a940f" gracePeriod=30
Oct 31 21:49:13.079596 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:13.079565    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:14.108829 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:14.108556    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:49:14.546518 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:14.546473    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:15.788099 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:15.788064    4460 patch_prober.go:29] interesting pod/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 21:49:15.788467 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:15.788117    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p" podUID="eb421e61-04b4-4c83-b872-b21c58b24504" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 21:49:16.355894 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:16.355863    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:18.190844 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.190803    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:18.376740 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.376712    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:49:18.476930 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.476880    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eb421e61-04b4-4c83-b872-b21c58b24504-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb421e61-04b4-4c83-b872-b21c58b24504-istiod-ca-cert\") pod \"eb421e61-04b4-4c83-b872-b21c58b24504\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") "
Oct 31 21:49:18.477139 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.477021    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-workload-certs\") pod \"eb421e61-04b4-4c83-b872-b21c58b24504\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") "
Oct 31 21:49:18.477139 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.477048    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-istio-data\") pod \"eb421e61-04b4-4c83-b872-b21c58b24504\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") "
Oct 31 21:49:18.477267 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.477184    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-istio-envoy\") pod \"eb421e61-04b4-4c83-b872-b21c58b24504\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") "
Oct 31 21:49:18.477267 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.477250    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-credential-socket\") pod \"eb421e61-04b4-4c83-b872-b21c58b24504\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") "
Oct 31 21:49:18.477386 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.477342    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-workload-certs" pod "eb421e61-04b4-4c83-b872-b21c58b24504" (UID: "eb421e61-04b4-4c83-b872-b21c58b24504"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:49:18.477386 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.477260    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb421e61-04b4-4c83-b872-b21c58b24504-istiod-ca-cert" pod "eb421e61-04b4-4c83-b872-b21c58b24504" (UID: "eb421e61-04b4-4c83-b872-b21c58b24504"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:49:18.477386 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.477347    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/eb421e61-04b4-4c83-b872-b21c58b24504-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb421e61-04b4-4c83-b872-b21c58b24504-istio-podinfo\") pod \"eb421e61-04b4-4c83-b872-b21c58b24504\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") "
Oct 31 21:49:18.477543 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.477448    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-workload-socket\") pod \"eb421e61-04b4-4c83-b872-b21c58b24504\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") "
Oct 31 21:49:18.477543 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.477484    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb421e61-04b4-4c83-b872-b21c58b24504-kube-api-access-fgvm5\" (UniqueName: \"kubernetes.io/projected/eb421e61-04b4-4c83-b872-b21c58b24504-kube-api-access-fgvm5\") pod \"eb421e61-04b4-4c83-b872-b21c58b24504\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") "
Oct 31 21:49:18.477654 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.477618    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb421e61-04b4-4c83-b872-b21c58b24504-istio-token\" (UniqueName: \"kubernetes.io/projected/eb421e61-04b4-4c83-b872-b21c58b24504-istio-token\") pod \"eb421e61-04b4-4c83-b872-b21c58b24504\" (UID: \"eb421e61-04b4-4c83-b872-b21c58b24504\") "
Oct 31 21:49:18.477654 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.477482    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-istio-data" pod "eb421e61-04b4-4c83-b872-b21c58b24504" (UID: "eb421e61-04b4-4c83-b872-b21c58b24504"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:49:18.477755 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.477613    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-credential-socket" pod "eb421e61-04b4-4c83-b872-b21c58b24504" (UID: "eb421e61-04b4-4c83-b872-b21c58b24504"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:49:18.478033 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.478006    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-workload-socket" pod "eb421e61-04b4-4c83-b872-b21c58b24504" (UID: "eb421e61-04b4-4c83-b872-b21c58b24504"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:49:18.478418 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.478397    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:49:18.478570 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.478553    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:49:18.478678 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.478664    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb421e61-04b4-4c83-b872-b21c58b24504-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:49:18.478777 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.478764    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:49:18.478874 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.478861    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:49:18.480052 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.480013    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-istio-envoy" pod "eb421e61-04b4-4c83-b872-b21c58b24504" (UID: "eb421e61-04b4-4c83-b872-b21c58b24504"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:49:18.480150 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.480072    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb421e61-04b4-4c83-b872-b21c58b24504-kube-api-access-fgvm5" pod "eb421e61-04b4-4c83-b872-b21c58b24504" (UID: "eb421e61-04b4-4c83-b872-b21c58b24504"). InnerVolumeSpecName "kube-api-access-fgvm5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:49:18.480248 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.480226    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eb421e61-04b4-4c83-b872-b21c58b24504-istio-podinfo" pod "eb421e61-04b4-4c83-b872-b21c58b24504" (UID: "eb421e61-04b4-4c83-b872-b21c58b24504"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:49:18.480525 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.480499    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb421e61-04b4-4c83-b872-b21c58b24504-istio-token" pod "eb421e61-04b4-4c83-b872-b21c58b24504" (UID: "eb421e61-04b4-4c83-b872-b21c58b24504"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:49:18.580286 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.580260    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgvm5\" (UniqueName: \"kubernetes.io/projected/eb421e61-04b4-4c83-b872-b21c58b24504-kube-api-access-fgvm5\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:49:18.580286 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.580286    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb421e61-04b4-4c83-b872-b21c58b24504-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:49:18.580432 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.580296    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb421e61-04b4-4c83-b872-b21c58b24504-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:49:18.580432 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.580304    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb421e61-04b4-4c83-b872-b21c58b24504-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:49:18.649203 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.649139    4460 generic.go:356] "Generic (PLEG): container finished" podID="eb421e61-04b4-4c83-b872-b21c58b24504" containerID="8f2f684c35a872d5d46d15ff905db809864f8ca44a3f2695bb9975a5480a940f" exitCode=0
Oct 31 21:49:18.649203 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.649185    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"
Oct 31 21:49:18.649351 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.649219    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p" event={"ID":"eb421e61-04b4-4c83-b872-b21c58b24504","Type":"ContainerDied","Data":"8f2f684c35a872d5d46d15ff905db809864f8ca44a3f2695bb9975a5480a940f"}
Oct 31 21:49:18.649351 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.649276    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p" event={"ID":"eb421e61-04b4-4c83-b872-b21c58b24504","Type":"ContainerDied","Data":"b952a9681d77e2e1c027b112d7e2f4390a99d2a3b7e4801068b7375093210fc5"}
Oct 31 21:49:18.649351 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.649303    4460 scope.go:122] "RemoveContainer" containerID="8f2f684c35a872d5d46d15ff905db809864f8ca44a3f2695bb9975a5480a940f"
Oct 31 21:49:18.663224 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.663202    4460 scope.go:122] "RemoveContainer" containerID="9a1f43ba4051c71ce2f9466c21e1925c9e6d7c500a7fae312b21df7af741577c"
Oct 31 21:49:18.672156 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.672136    4460 scope.go:122] "RemoveContainer" containerID="8f2f684c35a872d5d46d15ff905db809864f8ca44a3f2695bb9975a5480a940f"
Oct 31 21:49:18.672403 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:49:18.672381    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f2f684c35a872d5d46d15ff905db809864f8ca44a3f2695bb9975a5480a940f\": container with ID starting with 8f2f684c35a872d5d46d15ff905db809864f8ca44a3f2695bb9975a5480a940f not found: ID does not exist" containerID="8f2f684c35a872d5d46d15ff905db809864f8ca44a3f2695bb9975a5480a940f"
Oct 31 21:49:18.672483 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.672414    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f2f684c35a872d5d46d15ff905db809864f8ca44a3f2695bb9975a5480a940f"} err="failed to get container status \"8f2f684c35a872d5d46d15ff905db809864f8ca44a3f2695bb9975a5480a940f\": rpc error: code = NotFound desc = could not find container \"8f2f684c35a872d5d46d15ff905db809864f8ca44a3f2695bb9975a5480a940f\": container with ID starting with 8f2f684c35a872d5d46d15ff905db809864f8ca44a3f2695bb9975a5480a940f not found: ID does not exist"
Oct 31 21:49:18.672483 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.672438    4460 scope.go:122] "RemoveContainer" containerID="9a1f43ba4051c71ce2f9466c21e1925c9e6d7c500a7fae312b21df7af741577c"
Oct 31 21:49:18.672729 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:49:18.672711    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a1f43ba4051c71ce2f9466c21e1925c9e6d7c500a7fae312b21df7af741577c\": container with ID starting with 9a1f43ba4051c71ce2f9466c21e1925c9e6d7c500a7fae312b21df7af741577c not found: ID does not exist" containerID="9a1f43ba4051c71ce2f9466c21e1925c9e6d7c500a7fae312b21df7af741577c"
Oct 31 21:49:18.672773 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.672736    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a1f43ba4051c71ce2f9466c21e1925c9e6d7c500a7fae312b21df7af741577c"} err="failed to get container status \"9a1f43ba4051c71ce2f9466c21e1925c9e6d7c500a7fae312b21df7af741577c\": rpc error: code = NotFound desc = could not find container \"9a1f43ba4051c71ce2f9466c21e1925c9e6d7c500a7fae312b21df7af741577c\": container with ID starting with 9a1f43ba4051c71ce2f9466c21e1925c9e6d7c500a7fae312b21df7af741577c not found: ID does not exist"
Oct 31 21:49:18.707807 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.707778    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"]
Oct 31 21:49:18.711728 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:18.711708    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-b-ad121b-openshift-default-5b4f9b9589-25s5p"]
Oct 31 21:49:19.837375 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:19.837325    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:20.226580 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:20.226504    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eb421e61-04b4-4c83-b872-b21c58b24504" path="/var/lib/kubelet/pods/eb421e61-04b4-4c83-b872-b21c58b24504/volumes"
Oct 31 21:49:21.452882 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:49:21.452846    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:49:21.623917 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:21.623879    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:23.333877 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:23.333847    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:25.162445 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:25.162411    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:26.997238 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:26.997201    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:28.664283 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:28.664242    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:28.802922 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:28.802888    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:28.920414 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:28.920326    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:29.487515 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:29.487469    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:30.446113 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:30.446078    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:31.824874 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:31.824835    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:32.395350 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:32.395317    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:32.734638 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:32.734541    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:33.204210 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.204174    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"]
Oct 31 21:49:33.204819 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.204804    4460 cpu_manager.go:414] "containerMap: removing container" podUID="eb421e61-04b4-4c83-b872-b21c58b24504" containerName="istio-proxy"
Oct 31 21:49:33.204863 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.204820    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb421e61-04b4-4c83-b872-b21c58b24504" containerName="istio-proxy"
Oct 31 21:49:33.204863 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.204840    4460 cpu_manager.go:414] "containerMap: removing container" podUID="eb421e61-04b4-4c83-b872-b21c58b24504" containerName="istio-proxy"
Oct 31 21:49:33.204863 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.204845    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb421e61-04b4-4c83-b872-b21c58b24504" containerName="istio-proxy"
Oct 31 21:49:33.204981 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.204970    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="eb421e61-04b4-4c83-b872-b21c58b24504" containerName="istio-proxy"
Oct 31 21:49:33.208507 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.208491    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:49:33.226710 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.226684    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-efdee4-openshift-default-dockercfg-cvqkv\""
Oct 31 21:49:33.252180 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.252150    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:33.252989 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.252967    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"]
Oct 31 21:49:33.379894 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.379848    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-workload-socket\") pod \"e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") " pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:49:33.420966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.414081    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4f5j\" (UniqueName: \"kubernetes.io/projected/e424a250-e02c-48f5-b215-a37f7de92b50-kube-api-access-d4f5j\") pod \"e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") " pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:49:33.420966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.414247    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e424a250-e02c-48f5-b215-a37f7de92b50-istiod-ca-cert\") pod \"e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") " pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:49:33.420966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.414326    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e424a250-e02c-48f5-b215-a37f7de92b50-istio-token\") pod \"e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") " pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:49:33.420966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.414389    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-credential-socket\") pod \"e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") " pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:49:33.420966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.414438    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-istio-envoy\") pod \"e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") " pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:49:33.420966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.414584    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e424a250-e02c-48f5-b215-a37f7de92b50-istio-podinfo\") pod \"e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") " pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:49:33.420966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.414638    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-istio-data\") pod \"e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") " pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:49:33.420966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.414670    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-workload-certs\") pod \"e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") " pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:49:33.521706 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.521669    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-istio-envoy\") pod \"e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") " pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:49:33.521932 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.521785    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e424a250-e02c-48f5-b215-a37f7de92b50-istio-podinfo\") pod \"e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") " pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:49:33.521932 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.521823    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-istio-data\") pod \"e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") " pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:49:33.521932 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.521849    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-workload-certs\") pod \"e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") " pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:49:33.521932 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.521933    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-workload-socket\") pod \"e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") " pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:49:33.522210 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.522019    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4f5j\" (UniqueName: \"kubernetes.io/projected/e424a250-e02c-48f5-b215-a37f7de92b50-kube-api-access-d4f5j\") pod \"e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") " pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:49:33.522210 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.522133    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e424a250-e02c-48f5-b215-a37f7de92b50-istiod-ca-cert\") pod \"e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") " pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:49:33.523058 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.523032    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-istio-data\") pod \"e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") " pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:49:33.523740 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.523679    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-workload-socket\") pod \"e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") " pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:49:33.525358 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.524161    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-workload-certs\") pod \"e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") " pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:49:33.525615 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.524985    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e424a250-e02c-48f5-b215-a37f7de92b50-istiod-ca-cert\") pod \"e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") " pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:49:33.525795 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.525778    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e424a250-e02c-48f5-b215-a37f7de92b50-istio-token\") pod \"e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") " pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:49:33.526076 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.526054    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-credential-socket\") pod \"e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") " pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:49:33.526415 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.526393    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-credential-socket\") pod \"e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") " pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:49:33.530232 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.529978    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e424a250-e02c-48f5-b215-a37f7de92b50-istio-podinfo\") pod \"e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") " pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:49:33.530854 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.530619    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-istio-envoy\") pod \"e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") " pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:49:33.553277 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.553195    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4f5j\" (UniqueName: \"kubernetes.io/projected/e424a250-e02c-48f5-b215-a37f7de92b50-kube-api-access-d4f5j\") pod \"e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") " pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:49:33.557254 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.557224    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e424a250-e02c-48f5-b215-a37f7de92b50-istio-token\") pod \"e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") " pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:49:33.608309 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.608276    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:33.759679 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.759641    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:33.823552 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:33.823455    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:49:34.159081 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:34.159050    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"]
Oct 31 21:49:34.160531 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:49:34.160506    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode424a250_e02c_48f5_b215_a37f7de92b50.slice/crio-d1b2fae010c4b8f189a8083ddea51e079f5396a86cc57b1780284c32cdb5e265 WatchSource:0}: Error finding container d1b2fae010c4b8f189a8083ddea51e079f5396a86cc57b1780284c32cdb5e265: Status 404 returned error can't find the container with id d1b2fae010c4b8f189a8083ddea51e079f5396a86cc57b1780284c32cdb5e265
Oct 31 21:49:34.163356 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:34.163323    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:49:34.163436 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:34.163402    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:49:34.163483 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:34.163443    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:49:34.540140 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:34.540107    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:34.790529 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:34.790429    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp" event={"ID":"e424a250-e02c-48f5-b215-a37f7de92b50","Type":"ContainerStarted","Data":"66b37cf79b2f6cef0b5478c18ba9244b61b49f14ba60f45e55decfb3746370a3"}
Oct 31 21:49:34.790529 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:34.790508    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp" event={"ID":"e424a250-e02c-48f5-b215-a37f7de92b50","Type":"ContainerStarted","Data":"d1b2fae010c4b8f189a8083ddea51e079f5396a86cc57b1780284c32cdb5e265"}
Oct 31 21:49:34.825388 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:34.825350    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:49:34.865891 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:34.865834    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp" podStartSLOduration=1.865819218 podStartE2EDuration="1.865819218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:49:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:49:34.862135853 +0000 UTC m=+2014.778445285" watchObservedRunningTime="2026-10-31 21:49:34.865819218 +0000 UTC m=+2014.782128647"
Oct 31 21:49:35.829692 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:35.829664    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:49:36.543565 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:36.543536    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"]
Oct 31 21:49:36.543777 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:36.543746    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc" podUID="9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e" containerName="istio-proxy" containerID="cri-o://d4ad876cc03acf1f00d975c0660392bbf997eafbe355d59881fb056790c58be6" gracePeriod=30
Oct 31 21:49:36.630606 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:36.630564    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:36.803092 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:36.802983    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:49:36.804415 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:36.804393    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:49:37.366840 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:37.366774    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:37.473566 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:37.473532    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:38.536346 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:38.536309    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:39.125909 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:39.125875    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:39.248833 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:39.248797    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:39.319514 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:39.319483    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-efdee4-bf759994c-8q9gs"]
Oct 31 21:49:39.320284 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:39.320267    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="eb421e61-04b4-4c83-b872-b21c58b24504" containerName="istio-proxy"
Oct 31 21:49:39.324021 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:39.324001    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-efdee4-bf759994c-8q9gs"
Oct 31 21:49:39.333560 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:39.333363    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"]
Oct 31 21:49:39.333560 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:39.333526    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm" podUID="ae31c154-bb41-4ff0-aae3-93164dc757df" containerName="istio-proxy" containerID="cri-o://672fe7097d87651a9d95a6b43c036967cbd6bf929bcc17f04a0ca1dc35a4f4b9" gracePeriod=30
Oct 31 21:49:39.348204 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:39.348174    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-efdee4\""
Oct 31 21:49:39.352122 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:39.348427    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-efdee4-dockercfg-zlzkj\""
Oct 31 21:49:39.361670 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:39.361636    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-efdee4-bf759994c-8q9gs"]
Oct 31 21:49:39.405171 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:39.405094    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-efdee4-54d7b4cdff-2dr49"]
Oct 31 21:49:39.409572 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:39.409551    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-efdee4-54d7b4cdff-2dr49"
Oct 31 21:49:39.431140 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:39.431115    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-efdee4-54d7b4cdff-2dr49"]
Oct 31 21:49:39.433579 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:39.433553    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5gsw\" (UniqueName: \"kubernetes.io/projected/05a4225d-a8de-4e82-9b4d-ff819a2c5603-kube-api-access-j5gsw\") pod \"payload-pre-processing-e2e-explicit-efdee4-bf759994c-8q9gs\" (UID: \"05a4225d-a8de-4e82-9b4d-ff819a2c5603\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-efdee4-bf759994c-8q9gs"
Oct 31 21:49:39.433813 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:39.433785    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05a4225d-a8de-4e82-9b4d-ff819a2c5603-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-efdee4-bf759994c-8q9gs\" (UID: \"05a4225d-a8de-4e82-9b4d-ff819a2c5603\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-efdee4-bf759994c-8q9gs"
Oct 31 21:49:39.554013 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:39.537167    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5gsw\" (UniqueName: \"kubernetes.io/projected/05a4225d-a8de-4e82-9b4d-ff819a2c5603-kube-api-access-j5gsw\") pod \"payload-pre-processing-e2e-explicit-efdee4-bf759994c-8q9gs\" (UID: \"05a4225d-a8de-4e82-9b4d-ff819a2c5603\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-efdee4-bf759994c-8q9gs"
Oct 31 21:49:39.554754 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:39.554722    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18cfce38-8c7b-45be-b7e9-cdb545b104ec-plugins-config-volume\") pod \"payload-processing-e2e-explicit-efdee4-54d7b4cdff-2dr49\" (UID: \"18cfce38-8c7b-45be-b7e9-cdb545b104ec\") " pod="openshift-ingress/payload-processing-e2e-explicit-efdee4-54d7b4cdff-2dr49"
Oct 31 21:49:39.555012 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:39.554990    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7p7n8\" (UniqueName: \"kubernetes.io/projected/18cfce38-8c7b-45be-b7e9-cdb545b104ec-kube-api-access-7p7n8\") pod \"payload-processing-e2e-explicit-efdee4-54d7b4cdff-2dr49\" (UID: \"18cfce38-8c7b-45be-b7e9-cdb545b104ec\") " pod="openshift-ingress/payload-processing-e2e-explicit-efdee4-54d7b4cdff-2dr49"
Oct 31 21:49:39.558679 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:39.555253    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05a4225d-a8de-4e82-9b4d-ff819a2c5603-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-efdee4-bf759994c-8q9gs\" (UID: \"05a4225d-a8de-4e82-9b4d-ff819a2c5603\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-efdee4-bf759994c-8q9gs"
Oct 31 21:49:39.593380 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:39.556861    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05a4225d-a8de-4e82-9b4d-ff819a2c5603-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-efdee4-bf759994c-8q9gs\" (UID: \"05a4225d-a8de-4e82-9b4d-ff819a2c5603\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-efdee4-bf759994c-8q9gs"
Oct 31 21:49:39.701998 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:39.701573    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18cfce38-8c7b-45be-b7e9-cdb545b104ec-plugins-config-volume\") pod \"payload-processing-e2e-explicit-efdee4-54d7b4cdff-2dr49\" (UID: \"18cfce38-8c7b-45be-b7e9-cdb545b104ec\") " pod="openshift-ingress/payload-processing-e2e-explicit-efdee4-54d7b4cdff-2dr49"
Oct 31 21:49:39.701998 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:39.701650    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p7n8\" (UniqueName: \"kubernetes.io/projected/18cfce38-8c7b-45be-b7e9-cdb545b104ec-kube-api-access-7p7n8\") pod \"payload-processing-e2e-explicit-efdee4-54d7b4cdff-2dr49\" (UID: \"18cfce38-8c7b-45be-b7e9-cdb545b104ec\") " pod="openshift-ingress/payload-processing-e2e-explicit-efdee4-54d7b4cdff-2dr49"
Oct 31 21:49:39.702799 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:39.702776    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18cfce38-8c7b-45be-b7e9-cdb545b104ec-plugins-config-volume\") pod \"payload-processing-e2e-explicit-efdee4-54d7b4cdff-2dr49\" (UID: \"18cfce38-8c7b-45be-b7e9-cdb545b104ec\") " pod="openshift-ingress/payload-processing-e2e-explicit-efdee4-54d7b4cdff-2dr49"
Oct 31 21:49:39.704831 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:39.703498    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5gsw\" (UniqueName: \"kubernetes.io/projected/05a4225d-a8de-4e82-9b4d-ff819a2c5603-kube-api-access-j5gsw\") pod \"payload-pre-processing-e2e-explicit-efdee4-bf759994c-8q9gs\" (UID: \"05a4225d-a8de-4e82-9b4d-ff819a2c5603\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-efdee4-bf759994c-8q9gs"
Oct 31 21:49:39.727177 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:39.727142    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7p7n8\" (UniqueName: \"kubernetes.io/projected/18cfce38-8c7b-45be-b7e9-cdb545b104ec-kube-api-access-7p7n8\") pod \"payload-processing-e2e-explicit-efdee4-54d7b4cdff-2dr49\" (UID: \"18cfce38-8c7b-45be-b7e9-cdb545b104ec\") " pod="openshift-ingress/payload-processing-e2e-explicit-efdee4-54d7b4cdff-2dr49"
Oct 31 21:49:39.940808 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:39.940777    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-efdee4-bf759994c-8q9gs"
Oct 31 21:49:40.024984 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:40.024134    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-efdee4-54d7b4cdff-2dr49"
Oct 31 21:49:40.167018 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:40.166986    4460 patch_prober.go:29] interesting pod/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 21:49:40.167166 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:40.167037    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc" podUID="9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 21:49:40.249893 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:49:40.249865    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05a4225d_a8de_4e82_9b4d_ff819a2c5603.slice/crio-2f034e9b27795b1f781a68a33854277baa018a15a74390c51686432aae1f493a WatchSource:0}: Error finding container 2f034e9b27795b1f781a68a33854277baa018a15a74390c51686432aae1f493a: Status 404 returned error can't find the container with id 2f034e9b27795b1f781a68a33854277baa018a15a74390c51686432aae1f493a
Oct 31 21:49:40.250879 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:40.250853    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-efdee4-bf759994c-8q9gs"]
Oct 31 21:49:40.345733 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:40.345707    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-efdee4-54d7b4cdff-2dr49"]
Oct 31 21:49:40.345884 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:49:40.345863    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18cfce38_8c7b_45be_b7e9_cdb545b104ec.slice/crio-38c6e68bc67fcc89036cd0cb4f5fff3ff73873ed246dad2ff733f9b6c548d7de WatchSource:0}: Error finding container 38c6e68bc67fcc89036cd0cb4f5fff3ff73873ed246dad2ff733f9b6c548d7de: Status 404 returned error can't find the container with id 38c6e68bc67fcc89036cd0cb4f5fff3ff73873ed246dad2ff733f9b6c548d7de
Oct 31 21:49:40.417541 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:40.417513    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:40.829849 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:40.829807    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-efdee4-bf759994c-8q9gs" event={"ID":"05a4225d-a8de-4e82-9b4d-ff819a2c5603","Type":"ContainerStarted","Data":"dae84aa89216acc88efb5609a64f243de92b9da5275a0692cb9990e77ed21d0e"}
Oct 31 21:49:40.830284 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:40.829876    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-efdee4-bf759994c-8q9gs" event={"ID":"05a4225d-a8de-4e82-9b4d-ff819a2c5603","Type":"ContainerStarted","Data":"2f034e9b27795b1f781a68a33854277baa018a15a74390c51686432aae1f493a"}
Oct 31 21:49:40.830284 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:40.829910    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-efdee4-bf759994c-8q9gs"
Oct 31 21:49:40.831610 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:40.831587    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-efdee4-54d7b4cdff-2dr49" event={"ID":"18cfce38-8c7b-45be-b7e9-cdb545b104ec","Type":"ContainerStarted","Data":"85cd8a41a632632b6bc11382a0f9828462606b0abc1cd6cd2d0c21a23f94c58a"}
Oct 31 21:49:40.831726 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:40.831626    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-efdee4-54d7b4cdff-2dr49" event={"ID":"18cfce38-8c7b-45be-b7e9-cdb545b104ec","Type":"ContainerStarted","Data":"38c6e68bc67fcc89036cd0cb4f5fff3ff73873ed246dad2ff733f9b6c548d7de"}
Oct 31 21:49:40.831726 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:40.831655    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-efdee4-54d7b4cdff-2dr49"
Oct 31 21:49:40.889592 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:40.889543    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-efdee4-bf759994c-8q9gs" podStartSLOduration=1.889528699 podStartE2EDuration="1.889528699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:49:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:49:40.888592683 +0000 UTC m=+2020.804902113" watchObservedRunningTime="2026-10-31 21:49:40.889528699 +0000 UTC m=+2020.805838128"
Oct 31 21:49:40.938926 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:40.938877    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-efdee4-54d7b4cdff-2dr49" podStartSLOduration=1.938865556 podStartE2EDuration="1.938865556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:49:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:49:40.93779955 +0000 UTC m=+2020.854108981" watchObservedRunningTime="2026-10-31 21:49:40.938865556 +0000 UTC m=+2020.855174984"
Oct 31 21:49:41.840570 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:41.840541    4460 generic.go:356] "Generic (PLEG): container finished" podID="9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e" containerID="d4ad876cc03acf1f00d975c0660392bbf997eafbe355d59881fb056790c58be6" exitCode=0
Oct 31 21:49:41.840898 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:41.840614    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc" event={"ID":"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e","Type":"ContainerDied","Data":"d4ad876cc03acf1f00d975c0660392bbf997eafbe355d59881fb056790c58be6"}
Oct 31 21:49:41.840898 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:41.840697    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc" event={"ID":"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e","Type":"ContainerDied","Data":"f4281304c5f0e90a6a3b409a9360282f35c4d1e00f6633ae7d1a9d5de9a3f312"}
Oct 31 21:49:41.840898 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:41.840714    4460 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f4281304c5f0e90a6a3b409a9360282f35c4d1e00f6633ae7d1a9d5de9a3f312"
Oct 31 21:49:41.849846 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:41.849824    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:49:41.935954 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:41.935917    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istio-token\" (UniqueName: \"kubernetes.io/projected/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istio-token\") pod \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") "
Oct 31 21:49:41.936071 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:41.935991    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istio-podinfo\") pod \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") "
Oct 31 21:49:41.936071 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:41.936049    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istio-data\") pod \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") "
Oct 31 21:49:41.936154 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:41.936109    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-credential-socket\") pod \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") "
Oct 31 21:49:41.936198 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:41.936182    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istiod-ca-cert\") pod \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") "
Oct 31 21:49:41.936258 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:41.936220    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-workload-socket\") pod \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") "
Oct 31 21:49:41.936398 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:41.936345    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istio-data" pod "9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e" (UID: "9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:49:41.936398 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:41.936375    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-kube-api-access-lp959\" (UniqueName: \"kubernetes.io/projected/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-kube-api-access-lp959\") pod \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") "
Oct 31 21:49:41.936587 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:41.936423    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-credential-socket" pod "9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e" (UID: "9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:49:41.936587 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:41.936461    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-workload-socket" pod "9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e" (UID: "9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:49:41.936587 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:41.936514    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-workload-certs\") pod \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") "
Oct 31 21:49:41.936587 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:41.936542    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istio-envoy\") pod \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\" (UID: \"9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e\") "
Oct 31 21:49:41.936847 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:41.936822    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-workload-certs" pod "9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e" (UID: "9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:49:41.936998 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:41.936967    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istiod-ca-cert" pod "9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e" (UID: "9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:49:41.937531 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:41.937506    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:49:41.937628 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:41.937538    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:49:41.937628 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:41.937552    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:49:41.937628 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:41.937565    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:49:41.937628 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:41.937581    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:49:41.938679 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:41.938657    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istio-podinfo" pod "9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e" (UID: "9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:49:41.938863 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:41.938833    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istio-token" pod "9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e" (UID: "9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:49:41.939261 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:41.939243    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istio-envoy" pod "9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e" (UID: "9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:49:41.939785 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:41.939765    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-kube-api-access-lp959" pod "9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e" (UID: "9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e"). InnerVolumeSpecName "kube-api-access-lp959". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:49:42.038688 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:42.038656    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lp959\" (UniqueName: \"kubernetes.io/projected/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-kube-api-access-lp959\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:49:42.038688 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:42.038684    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:49:42.038688 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:42.038695    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:49:42.038982 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:42.038703    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:49:42.225614 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:42.225532    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:42.847190 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:42.847123    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"
Oct 31 21:49:42.924659 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:42.924625    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"]
Oct 31 21:49:42.938089 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:42.938062    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-j5gwc"]
Oct 31 21:49:43.934191 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:43.934152    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:44.226290 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.226203    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e" path="/var/lib/kubelet/pods/9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e/volumes"
Oct 31 21:49:44.598871 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.598843    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:49:44.671702 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.671670    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-istio-data\") pod \"ae31c154-bb41-4ff0-aae3-93164dc757df\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") "
Oct 31 21:49:44.671877 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.671717    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-workload-certs\") pod \"ae31c154-bb41-4ff0-aae3-93164dc757df\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") "
Oct 31 21:49:44.671877 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.671853    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ae31c154-bb41-4ff0-aae3-93164dc757df-kube-api-access-2qt8c\" (UniqueName: \"kubernetes.io/projected/ae31c154-bb41-4ff0-aae3-93164dc757df-kube-api-access-2qt8c\") pod \"ae31c154-bb41-4ff0-aae3-93164dc757df\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") "
Oct 31 21:49:44.672041 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.671915    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-credential-socket\") pod \"ae31c154-bb41-4ff0-aae3-93164dc757df\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") "
Oct 31 21:49:44.672041 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.672012    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ae31c154-bb41-4ff0-aae3-93164dc757df-istio-token\" (UniqueName: \"kubernetes.io/projected/ae31c154-bb41-4ff0-aae3-93164dc757df-istio-token\") pod \"ae31c154-bb41-4ff0-aae3-93164dc757df\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") "
Oct 31 21:49:44.672149 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.672022    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-istio-data" pod "ae31c154-bb41-4ff0-aae3-93164dc757df" (UID: "ae31c154-bb41-4ff0-aae3-93164dc757df"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:49:44.672149 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.672105    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-workload-certs" pod "ae31c154-bb41-4ff0-aae3-93164dc757df" (UID: "ae31c154-bb41-4ff0-aae3-93164dc757df"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:49:44.672239 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.672208    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-istio-envoy\") pod \"ae31c154-bb41-4ff0-aae3-93164dc757df\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") "
Oct 31 21:49:44.672325 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.672310    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-workload-socket\") pod \"ae31c154-bb41-4ff0-aae3-93164dc757df\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") "
Oct 31 21:49:44.672382 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.672349    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ae31c154-bb41-4ff0-aae3-93164dc757df-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae31c154-bb41-4ff0-aae3-93164dc757df-istiod-ca-cert\") pod \"ae31c154-bb41-4ff0-aae3-93164dc757df\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") "
Oct 31 21:49:44.672435 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.672372    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-credential-socket" pod "ae31c154-bb41-4ff0-aae3-93164dc757df" (UID: "ae31c154-bb41-4ff0-aae3-93164dc757df"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:49:44.672435 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.672397    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ae31c154-bb41-4ff0-aae3-93164dc757df-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae31c154-bb41-4ff0-aae3-93164dc757df-istio-podinfo\") pod \"ae31c154-bb41-4ff0-aae3-93164dc757df\" (UID: \"ae31c154-bb41-4ff0-aae3-93164dc757df\") "
Oct 31 21:49:44.673512 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.673051    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:49:44.673512 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.673077    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:49:44.673512 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.673092    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:49:44.673512 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.673111    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-workload-socket" pod "ae31c154-bb41-4ff0-aae3-93164dc757df" (UID: "ae31c154-bb41-4ff0-aae3-93164dc757df"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:49:44.673512 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.673468    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae31c154-bb41-4ff0-aae3-93164dc757df-istiod-ca-cert" pod "ae31c154-bb41-4ff0-aae3-93164dc757df" (UID: "ae31c154-bb41-4ff0-aae3-93164dc757df"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:49:44.675468 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.675439    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae31c154-bb41-4ff0-aae3-93164dc757df-istio-token" pod "ae31c154-bb41-4ff0-aae3-93164dc757df" (UID: "ae31c154-bb41-4ff0-aae3-93164dc757df"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:49:44.675571 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.675469    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae31c154-bb41-4ff0-aae3-93164dc757df-kube-api-access-2qt8c" pod "ae31c154-bb41-4ff0-aae3-93164dc757df" (UID: "ae31c154-bb41-4ff0-aae3-93164dc757df"). InnerVolumeSpecName "kube-api-access-2qt8c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:49:44.675785 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.675765    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ae31c154-bb41-4ff0-aae3-93164dc757df-istio-podinfo" pod "ae31c154-bb41-4ff0-aae3-93164dc757df" (UID: "ae31c154-bb41-4ff0-aae3-93164dc757df"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:49:44.676764 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.676745    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-istio-envoy" pod "ae31c154-bb41-4ff0-aae3-93164dc757df" (UID: "ae31c154-bb41-4ff0-aae3-93164dc757df"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:49:44.777565 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.777185    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:49:44.777565 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.777226    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae31c154-bb41-4ff0-aae3-93164dc757df-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:49:44.777565 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.777240    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae31c154-bb41-4ff0-aae3-93164dc757df-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:49:44.777565 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.777254    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae31c154-bb41-4ff0-aae3-93164dc757df-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:49:44.777565 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.777267    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qt8c\" (UniqueName: \"kubernetes.io/projected/ae31c154-bb41-4ff0-aae3-93164dc757df-kube-api-access-2qt8c\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:49:44.777565 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.777280    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae31c154-bb41-4ff0-aae3-93164dc757df-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:49:44.862242 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.862209    4460 generic.go:356] "Generic (PLEG): container finished" podID="ae31c154-bb41-4ff0-aae3-93164dc757df" containerID="672fe7097d87651a9d95a6b43c036967cbd6bf929bcc17f04a0ca1dc35a4f4b9" exitCode=0
Oct 31 21:49:44.862423 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.862268    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"
Oct 31 21:49:44.862423 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.862267    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm" event={"ID":"ae31c154-bb41-4ff0-aae3-93164dc757df","Type":"ContainerDied","Data":"672fe7097d87651a9d95a6b43c036967cbd6bf929bcc17f04a0ca1dc35a4f4b9"}
Oct 31 21:49:44.862423 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.862387    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm" event={"ID":"ae31c154-bb41-4ff0-aae3-93164dc757df","Type":"ContainerDied","Data":"6156aa29791be5302e7b26da7949c2c2bbe780683659aab95db49f17d4722300"}
Oct 31 21:49:44.862423 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.862411    4460 scope.go:122] "RemoveContainer" containerID="672fe7097d87651a9d95a6b43c036967cbd6bf929bcc17f04a0ca1dc35a4f4b9"
Oct 31 21:49:44.876182 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.876166    4460 scope.go:122] "RemoveContainer" containerID="672fe7097d87651a9d95a6b43c036967cbd6bf929bcc17f04a0ca1dc35a4f4b9"
Oct 31 21:49:44.876453 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:49:44.876435    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"672fe7097d87651a9d95a6b43c036967cbd6bf929bcc17f04a0ca1dc35a4f4b9\": container with ID starting with 672fe7097d87651a9d95a6b43c036967cbd6bf929bcc17f04a0ca1dc35a4f4b9 not found: ID does not exist" containerID="672fe7097d87651a9d95a6b43c036967cbd6bf929bcc17f04a0ca1dc35a4f4b9"
Oct 31 21:49:44.876519 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.876464    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"672fe7097d87651a9d95a6b43c036967cbd6bf929bcc17f04a0ca1dc35a4f4b9"} err="failed to get container status \"672fe7097d87651a9d95a6b43c036967cbd6bf929bcc17f04a0ca1dc35a4f4b9\": rpc error: code = NotFound desc = could not find container \"672fe7097d87651a9d95a6b43c036967cbd6bf929bcc17f04a0ca1dc35a4f4b9\": container with ID starting with 672fe7097d87651a9d95a6b43c036967cbd6bf929bcc17f04a0ca1dc35a4f4b9 not found: ID does not exist"
Oct 31 21:49:44.921671 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.921535    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"]
Oct 31 21:49:44.929375 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:44.929338    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-94gsm"]
Oct 31 21:49:45.716172 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:45.716127    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:46.225793 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:46.225759    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ae31c154-bb41-4ff0-aae3-93164dc757df" path="/var/lib/kubelet/pods/ae31c154-bb41-4ff0-aae3-93164dc757df/volumes"
Oct 31 21:49:47.397387 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:47.397347    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:47.645354 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:47.645321    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:48.993016 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:48.992979    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:50.711241 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:50.711202    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:51.452121 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:49:51.452086    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:49:51.843697 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:51.843670    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-efdee4-54d7b4cdff-2dr49"
Oct 31 21:49:51.844208 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:51.843732    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-efdee4-bf759994c-8q9gs"
Oct 31 21:49:52.354157 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:52.354120    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:54.031293 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:54.031262    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:55.044002 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:55.043967    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:49:55.846339 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:55.846297    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:56.873754 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:56.873715    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:57.742965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:57.742446    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:59.546458 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:59.546421    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:49:59.946612 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:49:59.946528    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:00.834571 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:00.834539    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:00.946388 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:00.946357    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:01.199084 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.199006    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:01.311174 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.311139    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"]
Oct 31 21:50:01.311788 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.311774    4460 cpu_manager.go:414] "containerMap: removing container" podUID="9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e" containerName="istio-proxy"
Oct 31 21:50:01.311833 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.311790    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e" containerName="istio-proxy"
Oct 31 21:50:01.311833 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.311818    4460 cpu_manager.go:414] "containerMap: removing container" podUID="ae31c154-bb41-4ff0-aae3-93164dc757df" containerName="istio-proxy"
Oct 31 21:50:01.311833 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.311823    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ae31c154-bb41-4ff0-aae3-93164dc757df" containerName="istio-proxy"
Oct 31 21:50:01.311928 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.311923    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="9410a6ce-b00f-4f8e-bdba-7a01cc4f4f9e" containerName="istio-proxy"
Oct 31 21:50:01.311981 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.311949    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="ae31c154-bb41-4ff0-aae3-93164dc757df" containerName="istio-proxy"
Oct 31 21:50:01.316194 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.316173    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 21:50:01.350904 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.350873    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:01.352650 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.352621    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"]
Oct 31 21:50:01.495045 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.493375    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:01.584841 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.584808    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-workload-certs\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 21:50:01.585105 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.585078    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istio-token\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 21:50:01.585209 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.585121    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istio-podinfo\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 21:50:01.585209 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.585166    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-workload-socket\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 21:50:01.585328 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.585215    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istiod-ca-cert\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 21:50:01.585328 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.585308    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istio-data\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 21:50:01.585425 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.585401    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istio-envoy\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 21:50:01.585488 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.585431    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8gw2\" (UniqueName: \"kubernetes.io/projected/d8dbe7b2-34ce-4590-825e-36a108f86b9f-kube-api-access-m8gw2\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 21:50:01.585546 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.585489    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-credential-socket\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 21:50:01.687957 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.687911    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-workload-certs\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 21:50:01.688163 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.687973    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istio-token\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 21:50:01.688163 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.688000    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istio-podinfo\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 21:50:01.688163 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.688034    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-workload-socket\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 21:50:01.688163 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.688076    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istiod-ca-cert\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 21:50:01.688163 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.688158    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istio-data\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 21:50:01.688424 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.688231    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istio-envoy\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 21:50:01.688424 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.688260    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8gw2\" (UniqueName: \"kubernetes.io/projected/d8dbe7b2-34ce-4590-825e-36a108f86b9f-kube-api-access-m8gw2\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 21:50:01.688424 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.688316    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-credential-socket\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 21:50:01.688572 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.688314    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-workload-certs\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 21:50:01.688780 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.688758    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istiod-ca-cert\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 21:50:01.688923 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.688768    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istio-data\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 21:50:01.689020 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.688925    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-workload-socket\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 21:50:01.689089 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.689024    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-credential-socket\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 21:50:01.691122 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.691101    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istio-envoy\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 21:50:01.691397 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.691378    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istio-podinfo\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 21:50:01.714317 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.714287    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istio-token\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 21:50:01.714424 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.714401    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8gw2\" (UniqueName: \"kubernetes.io/projected/d8dbe7b2-34ce-4590-825e-36a108f86b9f-kube-api-access-m8gw2\") pod \"e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") " pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 21:50:01.971269 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:01.971233    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 21:50:02.447376 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:02.447347    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"]
Oct 31 21:50:02.447592 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:02.447368    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:50:02.447796 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:02.447775    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:50:02.447992 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:02.447970    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:50:03.023793 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:03.023756    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:03.081839 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:03.081795    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj" event={"ID":"d8dbe7b2-34ce-4590-825e-36a108f86b9f","Type":"ContainerStarted","Data":"0e09d03a72a23024608dc7d8a250b7df63acdf799cd131571b9cd00f082cefd0"}
Oct 31 21:50:03.082060 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:03.081902    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj" event={"ID":"d8dbe7b2-34ce-4590-825e-36a108f86b9f","Type":"ContainerStarted","Data":"42a3b5ad8355c09173d59f9670bf70521b2b6decc7014df23f3370fe918388f9"}
Oct 31 21:50:03.189796 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:03.189721    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj" podStartSLOduration=2.189698121 podStartE2EDuration="2.189698121s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:50:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:50:03.152479138 +0000 UTC m=+2043.068788601" watchObservedRunningTime="2026-10-31 21:50:03.189698121 +0000 UTC m=+2043.106007551"
Oct 31 21:50:03.762340 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:03.762300    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:03.914236 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:03.914203    4460 scope.go:122] "RemoveContainer" containerID="d4ad876cc03acf1f00d975c0660392bbf997eafbe355d59881fb056790c58be6"
Oct 31 21:50:04.011010 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:04.005784    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 21:50:04.011209 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:04.011027    4460 patch_prober.go:29] interesting pod/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused" start-of-body=
Oct 31 21:50:04.011209 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:04.011104    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj" podUID="d8dbe7b2-34ce-4590-825e-36a108f86b9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused"
Oct 31 21:50:04.789509 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:04.789476    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:04.972332 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:04.972296    4460 patch_prober.go:29] interesting pod/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused" start-of-body=
Oct 31 21:50:04.972512 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:04.972351    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj" podUID="d8dbe7b2-34ce-4590-825e-36a108f86b9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused"
Oct 31 21:50:05.544998 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:05.542026    4460 patch_prober.go:29] interesting pod/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.148:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 21:50:05.544998 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:05.542089    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp" podUID="dd21505e-c4de-43eb-a5da-5c69c7c90785" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.148:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:50:05.971806 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:05.971721    4460 patch_prober.go:29] interesting pod/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused" start-of-body=
Oct 31 21:50:05.971806 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:05.971779    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj" podUID="d8dbe7b2-34ce-4590-825e-36a108f86b9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused"
Oct 31 21:50:06.547655 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:06.547613    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:06.972545 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:06.972462    4460 patch_prober.go:29] interesting pod/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused" start-of-body=
Oct 31 21:50:06.972545 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:06.972521    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj" podUID="d8dbe7b2-34ce-4590-825e-36a108f86b9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused"
Oct 31 21:50:07.971985 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:07.971889    4460 patch_prober.go:29] interesting pod/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused" start-of-body=
Oct 31 21:50:07.972195 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:07.971984    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj" podUID="d8dbe7b2-34ce-4590-825e-36a108f86b9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused"
Oct 31 21:50:08.168067 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:08.167978    4460 generic.go:356] "Generic (PLEG): container finished" podID="dd21505e-c4de-43eb-a5da-5c69c7c90785" containerID="0012406ddf0805880548028d005622769699f85c53a707bed83322e99b8f90e2" exitCode=0
Oct 31 21:50:08.168067 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:08.168026    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp" event={"ID":"dd21505e-c4de-43eb-a5da-5c69c7c90785","Type":"ContainerDied","Data":"0012406ddf0805880548028d005622769699f85c53a707bed83322e99b8f90e2"}
Oct 31 21:50:08.168585 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:08.168567    4460 scope.go:122] "RemoveContainer" containerID="0012406ddf0805880548028d005622769699f85c53a707bed83322e99b8f90e2"
Oct 31 21:50:08.170670 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:08.170637    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:50:08.170826 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:08.170720    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:50:08.170826 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:08.170775    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:50:08.250964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:08.250904    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:08.972574 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:08.972535    4460 patch_prober.go:29] interesting pod/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused" start-of-body=
Oct 31 21:50:08.972762 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:08.972596    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj" podUID="d8dbe7b2-34ce-4590-825e-36a108f86b9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.152:15021/healthz/ready\": dial tcp 10.133.0.152:15021: connect: connection refused"
Oct 31 21:50:09.179037 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:09.178986    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp" event={"ID":"dd21505e-c4de-43eb-a5da-5c69c7c90785","Type":"ContainerStarted","Data":"4fa133c03e4b25c0f40660b51016cd018a651e4af7b98d465398898c656dd3de"}
Oct 31 21:50:09.938535 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:09.938491    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:10.016751 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:10.016713    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 21:50:10.017635 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:10.017601    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 21:50:10.018735 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:10.018712    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 21:50:10.048431 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:10.048392    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:50:10.048596 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:10.048523    4460 patch_prober.go:29] interesting pod/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused" start-of-body=
Oct 31 21:50:10.048596 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:10.048579    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp" podUID="dd21505e-c4de-43eb-a5da-5c69c7c90785" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused"
Oct 31 21:50:11.048592 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:11.048554    4460 patch_prober.go:29] interesting pod/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused" start-of-body=
Oct 31 21:50:11.049007 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:11.048612    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp" podUID="dd21505e-c4de-43eb-a5da-5c69c7c90785" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused"
Oct 31 21:50:11.646695 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:11.646655    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:12.048491 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:12.048453    4460 patch_prober.go:29] interesting pod/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused" start-of-body=
Oct 31 21:50:12.048696 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:12.048549    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp" podUID="dd21505e-c4de-43eb-a5da-5c69c7c90785" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused"
Oct 31 21:50:13.056500 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:13.056431    4460 patch_prober.go:29] interesting pod/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused" start-of-body=
Oct 31 21:50:13.057171 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:13.057138    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp" podUID="dd21505e-c4de-43eb-a5da-5c69c7c90785" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused"
Oct 31 21:50:13.302805 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:13.302766    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:14.048923 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:14.048887    4460 patch_prober.go:29] interesting pod/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused" start-of-body=
Oct 31 21:50:14.049247 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:14.049200    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp" podUID="dd21505e-c4de-43eb-a5da-5c69c7c90785" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.148:15021/healthz/ready\": dial tcp 10.133.0.148:15021: connect: connection refused"
Oct 31 21:50:15.053201 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:15.053171    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:50:15.053716 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:15.053702    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:50:15.054430 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:15.054408    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 21:50:15.078780 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:15.078747    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:16.854139 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:16.854069    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:18.546436 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:18.546382    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:20.258258 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:20.258225    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:21.453239 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:50:21.453198    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:50:22.009579 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:22.009543    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:22.844131 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:22.844079    4460 patch_prober.go:29] interesting pod/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 21:50:22.846453 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:22.844171    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp" podUID="e424a250-e02c-48f5-b215-a37f7de92b50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 21:50:23.333461 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:23.333423    4460 generic.go:356] "Generic (PLEG): container finished" podID="e424a250-e02c-48f5-b215-a37f7de92b50" containerID="66b37cf79b2f6cef0b5478c18ba9244b61b49f14ba60f45e55decfb3746370a3" exitCode=0
Oct 31 21:50:23.333461 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:23.333449    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp" event={"ID":"e424a250-e02c-48f5-b215-a37f7de92b50","Type":"ContainerDied","Data":"66b37cf79b2f6cef0b5478c18ba9244b61b49f14ba60f45e55decfb3746370a3"}
Oct 31 21:50:23.333788 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:23.333773    4460 scope.go:122] "RemoveContainer" containerID="66b37cf79b2f6cef0b5478c18ba9244b61b49f14ba60f45e55decfb3746370a3"
Oct 31 21:50:23.335200 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:23.335177    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:50:23.335261 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:23.335240    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:50:23.335344 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:23.335270    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:50:23.695484 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:23.695390    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:24.358571 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:24.358540    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp" event={"ID":"e424a250-e02c-48f5-b215-a37f7de92b50","Type":"ContainerStarted","Data":"5d861c1b0b47dc16c28e8c4999955cfd9299694782ae9285fdcf8620964eed27"}
Oct 31 21:50:24.833969 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:24.831345    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:50:24.833969 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:24.831476    4460 patch_prober.go:29] interesting pod/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Oct 31 21:50:24.833969 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:24.831520    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp" podUID="e424a250-e02c-48f5-b215-a37f7de92b50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Oct 31 21:50:25.396239 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:25.396205    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:25.835965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:25.825294    4460 patch_prober.go:29] interesting pod/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Oct 31 21:50:25.835965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:25.825357    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp" podUID="e424a250-e02c-48f5-b215-a37f7de92b50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Oct 31 21:50:26.831964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:26.824273    4460 patch_prober.go:29] interesting pod/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Oct 31 21:50:26.831964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:26.824333    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp" podUID="e424a250-e02c-48f5-b215-a37f7de92b50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Oct 31 21:50:27.079225 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:27.079191    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:27.826964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:27.824264    4460 patch_prober.go:29] interesting pod/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Oct 31 21:50:27.826964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:27.824325    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp" podUID="e424a250-e02c-48f5-b215-a37f7de92b50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Oct 31 21:50:28.836465 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:28.836372    4460 patch_prober.go:29] interesting pod/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused" start-of-body=
Oct 31 21:50:28.837131 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:28.837095    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp" podUID="e424a250-e02c-48f5-b215-a37f7de92b50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.149:15021/healthz/ready\": dial tcp 10.133.0.149:15021: connect: connection refused"
Oct 31 21:50:28.947964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:28.947163    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:29.266362 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:29.266332    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58"]
Oct 31 21:50:29.271750 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:29.271724    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58"
Oct 31 21:50:29.291806 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:29.291780    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58"]
Oct 31 21:50:29.301058 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:29.301036    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w1-fb32d4\"/\"openshift-service-ca.crt\""
Oct 31 21:50:29.301058 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:29.301046    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w1-fb32d4\"/\"kube-root-ca.crt\""
Oct 31 21:50:29.324106 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:29.324058    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:29.334342 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:29.334316    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w1-fb32d4\"/\"auth-test-model-fb32d4-kserve-self-signed-certs\""
Oct 31 21:50:29.334513 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:29.334369    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w1-fb32d4\"/\"default-dockercfg-vhzzj\""
Oct 31 21:50:29.396567 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:29.396536    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqhwx\" (UniqueName: \"kubernetes.io/projected/aed4a2f6-db43-4088-a694-b38716d998aa-kube-api-access-xqhwx\") pod \"auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58\" (UID: \"aed4a2f6-db43-4088-a694-b38716d998aa\") " pod="ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58"
Oct 31 21:50:29.396787 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:29.396607    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aed4a2f6-db43-4088-a694-b38716d998aa-dshm\") pod \"auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58\" (UID: \"aed4a2f6-db43-4088-a694-b38716d998aa\") " pod="ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58"
Oct 31 21:50:29.396787 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:29.396734    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aed4a2f6-db43-4088-a694-b38716d998aa-tls-certs\") pod \"auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58\" (UID: \"aed4a2f6-db43-4088-a694-b38716d998aa\") " pod="ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58"
Oct 31 21:50:29.396888 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:29.396793    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aed4a2f6-db43-4088-a694-b38716d998aa-home\") pod \"auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58\" (UID: \"aed4a2f6-db43-4088-a694-b38716d998aa\") " pod="ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58"
Oct 31 21:50:29.396971 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:29.396933    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aed4a2f6-db43-4088-a694-b38716d998aa-model-cache\") pod \"auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58\" (UID: \"aed4a2f6-db43-4088-a694-b38716d998aa\") " pod="ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58"
Oct 31 21:50:29.397113 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:29.397076    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aed4a2f6-db43-4088-a694-b38716d998aa-tmp-dir\") pod \"auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58\" (UID: \"aed4a2f6-db43-4088-a694-b38716d998aa\") " pod="ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58"
Oct 31 21:50:29.498443 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:29.498402    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aed4a2f6-db43-4088-a694-b38716d998aa-tmp-dir\") pod \"auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58\" (UID: \"aed4a2f6-db43-4088-a694-b38716d998aa\") " pod="ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58"
Oct 31 21:50:29.499282 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:29.499253    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqhwx\" (UniqueName: \"kubernetes.io/projected/aed4a2f6-db43-4088-a694-b38716d998aa-kube-api-access-xqhwx\") pod \"auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58\" (UID: \"aed4a2f6-db43-4088-a694-b38716d998aa\") " pod="ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58"
Oct 31 21:50:29.504617 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:29.504585    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aed4a2f6-db43-4088-a694-b38716d998aa-dshm\") pod \"auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58\" (UID: \"aed4a2f6-db43-4088-a694-b38716d998aa\") " pod="ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58"
Oct 31 21:50:29.505322 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:29.505301    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aed4a2f6-db43-4088-a694-b38716d998aa-tls-certs\") pod \"auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58\" (UID: \"aed4a2f6-db43-4088-a694-b38716d998aa\") " pod="ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58"
Oct 31 21:50:29.506011 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:29.505990    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aed4a2f6-db43-4088-a694-b38716d998aa-home\") pod \"auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58\" (UID: \"aed4a2f6-db43-4088-a694-b38716d998aa\") " pod="ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58"
Oct 31 21:50:29.506848 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:29.499113    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aed4a2f6-db43-4088-a694-b38716d998aa-tmp-dir\") pod \"auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58\" (UID: \"aed4a2f6-db43-4088-a694-b38716d998aa\") " pod="ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58"
Oct 31 21:50:29.507085 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:29.506682    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aed4a2f6-db43-4088-a694-b38716d998aa-home\") pod \"auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58\" (UID: \"aed4a2f6-db43-4088-a694-b38716d998aa\") " pod="ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58"
Oct 31 21:50:29.522863 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:29.522787    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aed4a2f6-db43-4088-a694-b38716d998aa-model-cache\") pod \"auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58\" (UID: \"aed4a2f6-db43-4088-a694-b38716d998aa\") " pod="ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58"
Oct 31 21:50:29.523572 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:29.523545    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aed4a2f6-db43-4088-a694-b38716d998aa-model-cache\") pod \"auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58\" (UID: \"aed4a2f6-db43-4088-a694-b38716d998aa\") " pod="ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58"
Oct 31 21:50:29.538968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:29.538693    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqhwx\" (UniqueName: \"kubernetes.io/projected/aed4a2f6-db43-4088-a694-b38716d998aa-kube-api-access-xqhwx\") pod \"auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58\" (UID: \"aed4a2f6-db43-4088-a694-b38716d998aa\") " pod="ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58"
Oct 31 21:50:29.558737 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:29.558693    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aed4a2f6-db43-4088-a694-b38716d998aa-tls-certs\") pod \"auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58\" (UID: \"aed4a2f6-db43-4088-a694-b38716d998aa\") " pod="ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58"
Oct 31 21:50:29.576362 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:29.576324    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aed4a2f6-db43-4088-a694-b38716d998aa-dshm\") pod \"auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58\" (UID: \"aed4a2f6-db43-4088-a694-b38716d998aa\") " pod="ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58"
Oct 31 21:50:29.593362 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:29.593329    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58"
Oct 31 21:50:29.828614 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:29.828540    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:50:29.828955 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:29.828921    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:50:29.829680 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:29.829646    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:50:29.985100 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:29.985064    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58"]
Oct 31 21:50:29.986577 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:50:29.986549    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaed4a2f6_db43_4088_a694_b38716d998aa.slice/crio-752e1b8ddca4c67bdc444cb549b5f549daa0d4d199d4b4539dc466eafd4fdda0 WatchSource:0}: Error finding container 752e1b8ddca4c67bdc444cb549b5f549daa0d4d199d4b4539dc466eafd4fdda0: Status 404 returned error can't find the container with id 752e1b8ddca4c67bdc444cb549b5f549daa0d4d199d4b4539dc466eafd4fdda0
Oct 31 21:50:30.406615 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:30.406579    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58" event={"ID":"aed4a2f6-db43-4088-a694-b38716d998aa","Type":"ContainerStarted","Data":"4ea5e31fe67f03ce4f12dfb2a801aabdc316fbf3f32b59e37eae88410c25a5e2"}
Oct 31 21:50:30.406913 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:30.406620    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58" event={"ID":"aed4a2f6-db43-4088-a694-b38716d998aa","Type":"ContainerStarted","Data":"752e1b8ddca4c67bdc444cb549b5f549daa0d4d199d4b4539dc466eafd4fdda0"}
Oct 31 21:50:30.465993 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:30.465916    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58" podStartSLOduration=1.465896022 podStartE2EDuration="1.465896022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:50:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:50:30.462185494 +0000 UTC m=+2070.378494925" watchObservedRunningTime="2026-10-31 21:50:30.465896022 +0000 UTC m=+2070.382205451"
Oct 31 21:50:30.598962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:30.595178    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58"
Oct 31 21:50:31.015587 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:31.015546    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:32.695513 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:32.695462    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:33.355565 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:33.355533    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:33.465755 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:33.465715    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:34.421154 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:34.421120    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:36.168569 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:36.168531    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:37.808332 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:37.808275    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:39.538463 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:39.538428    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:39.613352 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:39.606448    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58"
Oct 31 21:50:39.653261 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:39.653232    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58"
Oct 31 21:50:40.482481 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:40.482454    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58"
Oct 31 21:50:41.243985 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:41.243935    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:42.944441 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:42.944387    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:42.948525 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:42.948497    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:50:44.910751 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:44.910716    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:46.628495 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:46.628461    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:47.770694 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:47.770655    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:48.374861 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:48.374829    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:50.128166 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:50.128129    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:51.453081 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:50:51.453046    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:50:51.796952 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:51.796906    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:53.578486 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:53.578455    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:55.230639 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:55.230601    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:55.983683 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:55.983651    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:57.094488 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:57.094446    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:50:58.859543 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:50:58.859502    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:00.611333 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:00.611300    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ssl9_26069598-5dc4-4c96-b55f-dd2f09884419/ovn-acl-logging/0.log"
Oct 31 21:51:00.626398 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:00.626373    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ssl9_26069598-5dc4-4c96-b55f-dd2f09884419/ovn-acl-logging/0.log"
Oct 31 21:51:00.632568 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:00.632542    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:01.200592 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:01.200552    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c"]
Oct 31 21:51:01.206100 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:01.206072    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c"
Oct 31 21:51:01.225379 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:01.225357    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c"]
Oct 31 21:51:01.234550 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:01.234497    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-efdee4\"/\"openshift-service-ca.crt\""
Oct 31 21:51:01.234687 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:01.234601    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-efdee4\"/\"default-dockercfg-ld726\""
Oct 31 21:51:01.238069 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:01.238037    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-efdee4\"/\"explicit-model-efdee4-kserve-self-signed-certs\""
Oct 31 21:51:01.238213 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:01.238109    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-efdee4\"/\"kube-root-ca.crt\""
Oct 31 21:51:01.249443 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:01.249405    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:01.265555 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:01.265529    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n59rz\" (UniqueName: \"kubernetes.io/projected/0d608736-5306-4c4a-9c55-fa8ee13af3ff-kube-api-access-n59rz\") pod \"explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c\" (UID: \"0d608736-5306-4c4a-9c55-fa8ee13af3ff\") " pod="ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c"
Oct 31 21:51:01.265683 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:01.265582    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d608736-5306-4c4a-9c55-fa8ee13af3ff-home\") pod \"explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c\" (UID: \"0d608736-5306-4c4a-9c55-fa8ee13af3ff\") " pod="ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c"
Oct 31 21:51:01.265683 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:01.265609    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d608736-5306-4c4a-9c55-fa8ee13af3ff-dshm\") pod \"explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c\" (UID: \"0d608736-5306-4c4a-9c55-fa8ee13af3ff\") " pod="ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c"
Oct 31 21:51:01.265807 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:01.265747    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d608736-5306-4c4a-9c55-fa8ee13af3ff-tls-certs\") pod \"explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c\" (UID: \"0d608736-5306-4c4a-9c55-fa8ee13af3ff\") " pod="ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c"
Oct 31 21:51:01.265807 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:01.265771    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d608736-5306-4c4a-9c55-fa8ee13af3ff-tmp-dir\") pod \"explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c\" (UID: \"0d608736-5306-4c4a-9c55-fa8ee13af3ff\") " pod="ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c"
Oct 31 21:51:01.265902 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:01.265887    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d608736-5306-4c4a-9c55-fa8ee13af3ff-model-cache\") pod \"explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c\" (UID: \"0d608736-5306-4c4a-9c55-fa8ee13af3ff\") " pod="ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c"
Oct 31 21:51:01.366893 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:01.366858    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d608736-5306-4c4a-9c55-fa8ee13af3ff-model-cache\") pod \"explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c\" (UID: \"0d608736-5306-4c4a-9c55-fa8ee13af3ff\") " pod="ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c"
Oct 31 21:51:01.367145 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:01.367122    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n59rz\" (UniqueName: \"kubernetes.io/projected/0d608736-5306-4c4a-9c55-fa8ee13af3ff-kube-api-access-n59rz\") pod \"explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c\" (UID: \"0d608736-5306-4c4a-9c55-fa8ee13af3ff\") " pod="ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c"
Oct 31 21:51:01.367555 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:01.367218    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d608736-5306-4c4a-9c55-fa8ee13af3ff-home\") pod \"explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c\" (UID: \"0d608736-5306-4c4a-9c55-fa8ee13af3ff\") " pod="ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c"
Oct 31 21:51:01.367555 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:01.367257    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d608736-5306-4c4a-9c55-fa8ee13af3ff-dshm\") pod \"explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c\" (UID: \"0d608736-5306-4c4a-9c55-fa8ee13af3ff\") " pod="ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c"
Oct 31 21:51:01.367555 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:01.367278    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d608736-5306-4c4a-9c55-fa8ee13af3ff-model-cache\") pod \"explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c\" (UID: \"0d608736-5306-4c4a-9c55-fa8ee13af3ff\") " pod="ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c"
Oct 31 21:51:01.367555 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:01.367356    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d608736-5306-4c4a-9c55-fa8ee13af3ff-tls-certs\") pod \"explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c\" (UID: \"0d608736-5306-4c4a-9c55-fa8ee13af3ff\") " pod="ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c"
Oct 31 21:51:01.367555 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:01.367396    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d608736-5306-4c4a-9c55-fa8ee13af3ff-tmp-dir\") pod \"explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c\" (UID: \"0d608736-5306-4c4a-9c55-fa8ee13af3ff\") " pod="ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c"
Oct 31 21:51:01.367802 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:01.367568    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d608736-5306-4c4a-9c55-fa8ee13af3ff-home\") pod \"explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c\" (UID: \"0d608736-5306-4c4a-9c55-fa8ee13af3ff\") " pod="ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c"
Oct 31 21:51:01.367802 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:01.367696    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d608736-5306-4c4a-9c55-fa8ee13af3ff-tmp-dir\") pod \"explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c\" (UID: \"0d608736-5306-4c4a-9c55-fa8ee13af3ff\") " pod="ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c"
Oct 31 21:51:01.369419 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:01.369398    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d608736-5306-4c4a-9c55-fa8ee13af3ff-dshm\") pod \"explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c\" (UID: \"0d608736-5306-4c4a-9c55-fa8ee13af3ff\") " pod="ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c"
Oct 31 21:51:01.369697 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:01.369679    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d608736-5306-4c4a-9c55-fa8ee13af3ff-tls-certs\") pod \"explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c\" (UID: \"0d608736-5306-4c4a-9c55-fa8ee13af3ff\") " pod="ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c"
Oct 31 21:51:01.392047 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:01.392025    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n59rz\" (UniqueName: \"kubernetes.io/projected/0d608736-5306-4c4a-9c55-fa8ee13af3ff-kube-api-access-n59rz\") pod \"explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c\" (UID: \"0d608736-5306-4c4a-9c55-fa8ee13af3ff\") " pod="ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c"
Oct 31 21:51:01.528046 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:01.522212    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c"
Oct 31 21:51:01.891954 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:01.891911    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c"]
Oct 31 21:51:01.892639 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:51:01.892613    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0d608736_5306_4c4a_9c55_fa8ee13af3ff.slice/crio-03b21c1394280b74b5a36b3394aeb39ae33d9da3b305793a74e88f553605fada WatchSource:0}: Error finding container 03b21c1394280b74b5a36b3394aeb39ae33d9da3b305793a74e88f553605fada: Status 404 returned error can't find the container with id 03b21c1394280b74b5a36b3394aeb39ae33d9da3b305793a74e88f553605fada
Oct 31 21:51:02.458970 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:02.456203    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:02.626496 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:02.626455    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c" event={"ID":"0d608736-5306-4c4a-9c55-fa8ee13af3ff","Type":"ContainerStarted","Data":"cd4e623c3c042e394c5209004e882f0ac1fe71f8d8f96d2fb67be0224c620bbf"}
Oct 31 21:51:02.626496 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:02.626495    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c" event={"ID":"0d608736-5306-4c4a-9c55-fa8ee13af3ff","Type":"ContainerStarted","Data":"03b21c1394280b74b5a36b3394aeb39ae33d9da3b305793a74e88f553605fada"}
Oct 31 21:51:02.672469 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:02.672409    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c" podStartSLOduration=1.672392915 podStartE2EDuration="1.672392915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:51:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:51:02.671569957 +0000 UTC m=+2102.587879387" watchObservedRunningTime="2026-10-31 21:51:02.672392915 +0000 UTC m=+2102.588702346"
Oct 31 21:51:03.523627 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:03.523597    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c"
Oct 31 21:51:04.221983 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:04.221933    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:05.938194 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:05.938156    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:07.639634 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:07.639594    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:07.964426 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:07.964348    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c"]
Oct 31 21:51:08.010398 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:08.010369    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:08.412303 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:08.412257    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:09.536481 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:09.536445    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:11.327245 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:11.327205    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:13.025175 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:13.025099    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:14.968277 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:14.968234    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:16.830387 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:16.830345    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:18.615222 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:18.615179    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:18.793182 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:18.793152    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-efdee4-54d7b4cdff-2dr49"]
Oct 31 21:51:18.833476 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:18.833445    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-efdee4-bf759994c-8q9gs"]
Oct 31 21:51:18.945573 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:18.945474    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:19.079965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:19.070841    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:20.119573 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:20.119542    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:20.656170 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:20.656136    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:20.770865 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:20.770835    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:20.904503 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:20.903618    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:21.343734 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:21.343690    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:21.453340 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:51:21.453301    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:51:22.860264 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:22.860221    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:22.987407 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:22.987370    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c" podUID="0d608736-5306-4c4a-9c55-fa8ee13af3ff" containerName="main" containerID="cri-o://cd4e623c3c042e394c5209004e882f0ac1fe71f8d8f96d2fb67be0224c620bbf" gracePeriod=45
Oct 31 21:51:23.262522 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:23.262493    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c"
Oct 31 21:51:23.389677 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:23.389648    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0d608736-5306-4c4a-9c55-fa8ee13af3ff-tls-certs\" (UniqueName: \"kubernetes.io/secret/0d608736-5306-4c4a-9c55-fa8ee13af3ff-tls-certs\") pod \"0d608736-5306-4c4a-9c55-fa8ee13af3ff\" (UID: \"0d608736-5306-4c4a-9c55-fa8ee13af3ff\") "
Oct 31 21:51:23.389843 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:23.389687    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d608736-5306-4c4a-9c55-fa8ee13af3ff-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d608736-5306-4c4a-9c55-fa8ee13af3ff-tmp-dir\") pod \"0d608736-5306-4c4a-9c55-fa8ee13af3ff\" (UID: \"0d608736-5306-4c4a-9c55-fa8ee13af3ff\") "
Oct 31 21:51:23.389843 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:23.389802    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d608736-5306-4c4a-9c55-fa8ee13af3ff-dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d608736-5306-4c4a-9c55-fa8ee13af3ff-dshm\") pod \"0d608736-5306-4c4a-9c55-fa8ee13af3ff\" (UID: \"0d608736-5306-4c4a-9c55-fa8ee13af3ff\") "
Oct 31 21:51:23.389925 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:23.389889    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d608736-5306-4c4a-9c55-fa8ee13af3ff-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d608736-5306-4c4a-9c55-fa8ee13af3ff-model-cache\") pod \"0d608736-5306-4c4a-9c55-fa8ee13af3ff\" (UID: \"0d608736-5306-4c4a-9c55-fa8ee13af3ff\") "
Oct 31 21:51:23.390021 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:23.389999    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d608736-5306-4c4a-9c55-fa8ee13af3ff-home\" (UniqueName: \"kubernetes.io/empty-dir/0d608736-5306-4c4a-9c55-fa8ee13af3ff-home\") pod \"0d608736-5306-4c4a-9c55-fa8ee13af3ff\" (UID: \"0d608736-5306-4c4a-9c55-fa8ee13af3ff\") "
Oct 31 21:51:23.390021 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:23.390008    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d608736-5306-4c4a-9c55-fa8ee13af3ff-tmp-dir" pod "0d608736-5306-4c4a-9c55-fa8ee13af3ff" (UID: "0d608736-5306-4c4a-9c55-fa8ee13af3ff"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:51:23.390138 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:23.390075    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d608736-5306-4c4a-9c55-fa8ee13af3ff-kube-api-access-n59rz\" (UniqueName: \"kubernetes.io/projected/0d608736-5306-4c4a-9c55-fa8ee13af3ff-kube-api-access-n59rz\") pod \"0d608736-5306-4c4a-9c55-fa8ee13af3ff\" (UID: \"0d608736-5306-4c4a-9c55-fa8ee13af3ff\") "
Oct 31 21:51:23.390214 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:23.390169    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d608736-5306-4c4a-9c55-fa8ee13af3ff-model-cache" pod "0d608736-5306-4c4a-9c55-fa8ee13af3ff" (UID: "0d608736-5306-4c4a-9c55-fa8ee13af3ff"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:51:23.390264 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:23.390216    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d608736-5306-4c4a-9c55-fa8ee13af3ff-home" pod "0d608736-5306-4c4a-9c55-fa8ee13af3ff" (UID: "0d608736-5306-4c4a-9c55-fa8ee13af3ff"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:51:23.390836 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:23.390810    4460 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d608736-5306-4c4a-9c55-fa8ee13af3ff-model-cache\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:51:23.390836 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:23.390838    4460 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d608736-5306-4c4a-9c55-fa8ee13af3ff-home\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:51:23.391061 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:23.390852    4460 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d608736-5306-4c4a-9c55-fa8ee13af3ff-tmp-dir\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:51:23.392018 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:23.391996    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d608736-5306-4c4a-9c55-fa8ee13af3ff-dshm" pod "0d608736-5306-4c4a-9c55-fa8ee13af3ff" (UID: "0d608736-5306-4c4a-9c55-fa8ee13af3ff"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:51:23.392109 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:23.392048    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d608736-5306-4c4a-9c55-fa8ee13af3ff-tls-certs" pod "0d608736-5306-4c4a-9c55-fa8ee13af3ff" (UID: "0d608736-5306-4c4a-9c55-fa8ee13af3ff"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 21:51:23.392192 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:23.392177    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d608736-5306-4c4a-9c55-fa8ee13af3ff-kube-api-access-n59rz" pod "0d608736-5306-4c4a-9c55-fa8ee13af3ff" (UID: "0d608736-5306-4c4a-9c55-fa8ee13af3ff"). InnerVolumeSpecName "kube-api-access-n59rz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:51:23.492387 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:23.492330    4460 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d608736-5306-4c4a-9c55-fa8ee13af3ff-tls-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:51:23.492387 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:23.492352    4460 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d608736-5306-4c4a-9c55-fa8ee13af3ff-dshm\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:51:23.492387 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:23.492362    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n59rz\" (UniqueName: \"kubernetes.io/projected/0d608736-5306-4c4a-9c55-fa8ee13af3ff-kube-api-access-n59rz\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:51:23.793743 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:23.793706    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-efdee4-54d7b4cdff-2dr49" podUID="18cfce38-8c7b-45be-b7e9-cdb545b104ec" containerName="payload-processing" containerID="cri-o://85cd8a41a632632b6bc11382a0f9828462606b0abc1cd6cd2d0c21a23f94c58a" gracePeriod=25
Oct 31 21:51:23.817323 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:23.817293    4460 generic.go:356] "Generic (PLEG): container finished" podID="0d608736-5306-4c4a-9c55-fa8ee13af3ff" containerID="cd4e623c3c042e394c5209004e882f0ac1fe71f8d8f96d2fb67be0224c620bbf" exitCode=0
Oct 31 21:51:23.817446 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:23.817338    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c"
Oct 31 21:51:23.817446 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:23.817356    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c" event={"ID":"0d608736-5306-4c4a-9c55-fa8ee13af3ff","Type":"ContainerDied","Data":"cd4e623c3c042e394c5209004e882f0ac1fe71f8d8f96d2fb67be0224c620bbf"}
Oct 31 21:51:23.817446 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:23.817398    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c" event={"ID":"0d608736-5306-4c4a-9c55-fa8ee13af3ff","Type":"ContainerDied","Data":"03b21c1394280b74b5a36b3394aeb39ae33d9da3b305793a74e88f553605fada"}
Oct 31 21:51:23.817446 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:23.817422    4460 scope.go:122] "RemoveContainer" containerID="cd4e623c3c042e394c5209004e882f0ac1fe71f8d8f96d2fb67be0224c620bbf"
Oct 31 21:51:23.833868 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:23.833826    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-efdee4-bf759994c-8q9gs" podUID="05a4225d-a8de-4e82-9b4d-ff819a2c5603" containerName="payload-pre-processing" containerID="cri-o://dae84aa89216acc88efb5609a64f243de92b9da5275a0692cb9990e77ed21d0e" gracePeriod=25
Oct 31 21:51:23.910318 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:23.910296    4460 scope.go:122] "RemoveContainer" containerID="cd4e623c3c042e394c5209004e882f0ac1fe71f8d8f96d2fb67be0224c620bbf"
Oct 31 21:51:23.910667 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:51:23.910647    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd4e623c3c042e394c5209004e882f0ac1fe71f8d8f96d2fb67be0224c620bbf\": container with ID starting with cd4e623c3c042e394c5209004e882f0ac1fe71f8d8f96d2fb67be0224c620bbf not found: ID does not exist" containerID="cd4e623c3c042e394c5209004e882f0ac1fe71f8d8f96d2fb67be0224c620bbf"
Oct 31 21:51:23.910712 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:23.910674    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd4e623c3c042e394c5209004e882f0ac1fe71f8d8f96d2fb67be0224c620bbf"} err="failed to get container status \"cd4e623c3c042e394c5209004e882f0ac1fe71f8d8f96d2fb67be0224c620bbf\": rpc error: code = NotFound desc = could not find container \"cd4e623c3c042e394c5209004e882f0ac1fe71f8d8f96d2fb67be0224c620bbf\": container with ID starting with cd4e623c3c042e394c5209004e882f0ac1fe71f8d8f96d2fb67be0224c620bbf not found: ID does not exist"
Oct 31 21:51:23.951878 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:23.951834    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c"]
Oct 31 21:51:23.979590 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:23.979553    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-efdee4/explicit-model-efdee4-kserve-6fdc4cc76f-wnr7c"]
Oct 31 21:51:24.128871 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.128840    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-efdee4-54d7b4cdff-2dr49"
Oct 31 21:51:24.182372 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.182344    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-efdee4-bf759994c-8q9gs"
Oct 31 21:51:24.203256 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.203224    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/18cfce38-8c7b-45be-b7e9-cdb545b104ec-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18cfce38-8c7b-45be-b7e9-cdb545b104ec-plugins-config-volume\") pod \"18cfce38-8c7b-45be-b7e9-cdb545b104ec\" (UID: \"18cfce38-8c7b-45be-b7e9-cdb545b104ec\") "
Oct 31 21:51:24.203426 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.203389    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/18cfce38-8c7b-45be-b7e9-cdb545b104ec-kube-api-access-7p7n8\" (UniqueName: \"kubernetes.io/projected/18cfce38-8c7b-45be-b7e9-cdb545b104ec-kube-api-access-7p7n8\") pod \"18cfce38-8c7b-45be-b7e9-cdb545b104ec\" (UID: \"18cfce38-8c7b-45be-b7e9-cdb545b104ec\") "
Oct 31 21:51:24.203662 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.203634    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18cfce38-8c7b-45be-b7e9-cdb545b104ec-plugins-config-volume" pod "18cfce38-8c7b-45be-b7e9-cdb545b104ec" (UID: "18cfce38-8c7b-45be-b7e9-cdb545b104ec"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:51:24.204216 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.204186    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18cfce38-8c7b-45be-b7e9-cdb545b104ec-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:51:24.205524 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.205503    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18cfce38-8c7b-45be-b7e9-cdb545b104ec-kube-api-access-7p7n8" pod "18cfce38-8c7b-45be-b7e9-cdb545b104ec" (UID: "18cfce38-8c7b-45be-b7e9-cdb545b104ec"). InnerVolumeSpecName "kube-api-access-7p7n8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:51:24.236738 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.236702    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0d608736-5306-4c4a-9c55-fa8ee13af3ff" path="/var/lib/kubelet/pods/0d608736-5306-4c4a-9c55-fa8ee13af3ff/volumes"
Oct 31 21:51:24.305174 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.305146    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/05a4225d-a8de-4e82-9b4d-ff819a2c5603-kube-api-access-j5gsw\" (UniqueName: \"kubernetes.io/projected/05a4225d-a8de-4e82-9b4d-ff819a2c5603-kube-api-access-j5gsw\") pod \"05a4225d-a8de-4e82-9b4d-ff819a2c5603\" (UID: \"05a4225d-a8de-4e82-9b4d-ff819a2c5603\") "
Oct 31 21:51:24.305174 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.305178    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/05a4225d-a8de-4e82-9b4d-ff819a2c5603-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05a4225d-a8de-4e82-9b4d-ff819a2c5603-plugins-config-volume\") pod \"05a4225d-a8de-4e82-9b4d-ff819a2c5603\" (UID: \"05a4225d-a8de-4e82-9b4d-ff819a2c5603\") "
Oct 31 21:51:24.305623 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.305596    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05a4225d-a8de-4e82-9b4d-ff819a2c5603-plugins-config-volume" pod "05a4225d-a8de-4e82-9b4d-ff819a2c5603" (UID: "05a4225d-a8de-4e82-9b4d-ff819a2c5603"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:51:24.305779 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.305762    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7p7n8\" (UniqueName: \"kubernetes.io/projected/18cfce38-8c7b-45be-b7e9-cdb545b104ec-kube-api-access-7p7n8\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:51:24.305835 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.305785    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05a4225d-a8de-4e82-9b4d-ff819a2c5603-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:51:24.307189 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.307165    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05a4225d-a8de-4e82-9b4d-ff819a2c5603-kube-api-access-j5gsw" pod "05a4225d-a8de-4e82-9b4d-ff819a2c5603" (UID: "05a4225d-a8de-4e82-9b4d-ff819a2c5603"). InnerVolumeSpecName "kube-api-access-j5gsw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:51:24.407104 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.407030    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5gsw\" (UniqueName: \"kubernetes.io/projected/05a4225d-a8de-4e82-9b4d-ff819a2c5603-kube-api-access-j5gsw\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:51:24.612683 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.612628    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:24.826461 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.826429    4460 generic.go:356] "Generic (PLEG): container finished" podID="18cfce38-8c7b-45be-b7e9-cdb545b104ec" containerID="85cd8a41a632632b6bc11382a0f9828462606b0abc1cd6cd2d0c21a23f94c58a" exitCode=0
Oct 31 21:51:24.826660 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.826494    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-efdee4-54d7b4cdff-2dr49"
Oct 31 21:51:24.826660 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.826519    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-efdee4-54d7b4cdff-2dr49" event={"ID":"18cfce38-8c7b-45be-b7e9-cdb545b104ec","Type":"ContainerDied","Data":"85cd8a41a632632b6bc11382a0f9828462606b0abc1cd6cd2d0c21a23f94c58a"}
Oct 31 21:51:24.826660 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.826577    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-efdee4-54d7b4cdff-2dr49" event={"ID":"18cfce38-8c7b-45be-b7e9-cdb545b104ec","Type":"ContainerDied","Data":"38c6e68bc67fcc89036cd0cb4f5fff3ff73873ed246dad2ff733f9b6c548d7de"}
Oct 31 21:51:24.826660 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.826606    4460 scope.go:122] "RemoveContainer" containerID="85cd8a41a632632b6bc11382a0f9828462606b0abc1cd6cd2d0c21a23f94c58a"
Oct 31 21:51:24.828065 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.828039    4460 generic.go:356] "Generic (PLEG): container finished" podID="05a4225d-a8de-4e82-9b4d-ff819a2c5603" containerID="dae84aa89216acc88efb5609a64f243de92b9da5275a0692cb9990e77ed21d0e" exitCode=0
Oct 31 21:51:24.828174 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.828102    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-efdee4-bf759994c-8q9gs"
Oct 31 21:51:24.828174 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.828127    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-efdee4-bf759994c-8q9gs" event={"ID":"05a4225d-a8de-4e82-9b4d-ff819a2c5603","Type":"ContainerDied","Data":"dae84aa89216acc88efb5609a64f243de92b9da5275a0692cb9990e77ed21d0e"}
Oct 31 21:51:24.828298 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.828194    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-efdee4-bf759994c-8q9gs" event={"ID":"05a4225d-a8de-4e82-9b4d-ff819a2c5603","Type":"ContainerDied","Data":"2f034e9b27795b1f781a68a33854277baa018a15a74390c51686432aae1f493a"}
Oct 31 21:51:24.839849 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.839828    4460 scope.go:122] "RemoveContainer" containerID="85cd8a41a632632b6bc11382a0f9828462606b0abc1cd6cd2d0c21a23f94c58a"
Oct 31 21:51:24.840279 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:51:24.840256    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85cd8a41a632632b6bc11382a0f9828462606b0abc1cd6cd2d0c21a23f94c58a\": container with ID starting with 85cd8a41a632632b6bc11382a0f9828462606b0abc1cd6cd2d0c21a23f94c58a not found: ID does not exist" containerID="85cd8a41a632632b6bc11382a0f9828462606b0abc1cd6cd2d0c21a23f94c58a"
Oct 31 21:51:24.840395 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.840285    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85cd8a41a632632b6bc11382a0f9828462606b0abc1cd6cd2d0c21a23f94c58a"} err="failed to get container status \"85cd8a41a632632b6bc11382a0f9828462606b0abc1cd6cd2d0c21a23f94c58a\": rpc error: code = NotFound desc = could not find container \"85cd8a41a632632b6bc11382a0f9828462606b0abc1cd6cd2d0c21a23f94c58a\": container with ID starting with 85cd8a41a632632b6bc11382a0f9828462606b0abc1cd6cd2d0c21a23f94c58a not found: ID does not exist"
Oct 31 21:51:24.840395 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.840307    4460 scope.go:122] "RemoveContainer" containerID="dae84aa89216acc88efb5609a64f243de92b9da5275a0692cb9990e77ed21d0e"
Oct 31 21:51:24.850031 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.850015    4460 scope.go:122] "RemoveContainer" containerID="dae84aa89216acc88efb5609a64f243de92b9da5275a0692cb9990e77ed21d0e"
Oct 31 21:51:24.850303 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:51:24.850283    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dae84aa89216acc88efb5609a64f243de92b9da5275a0692cb9990e77ed21d0e\": container with ID starting with dae84aa89216acc88efb5609a64f243de92b9da5275a0692cb9990e77ed21d0e not found: ID does not exist" containerID="dae84aa89216acc88efb5609a64f243de92b9da5275a0692cb9990e77ed21d0e"
Oct 31 21:51:24.850356 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.850309    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dae84aa89216acc88efb5609a64f243de92b9da5275a0692cb9990e77ed21d0e"} err="failed to get container status \"dae84aa89216acc88efb5609a64f243de92b9da5275a0692cb9990e77ed21d0e\": rpc error: code = NotFound desc = could not find container \"dae84aa89216acc88efb5609a64f243de92b9da5275a0692cb9990e77ed21d0e\": container with ID starting with dae84aa89216acc88efb5609a64f243de92b9da5275a0692cb9990e77ed21d0e not found: ID does not exist"
Oct 31 21:51:24.888430 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.888402    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-efdee4-54d7b4cdff-2dr49"]
Oct 31 21:51:24.899426 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.899399    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-efdee4-54d7b4cdff-2dr49"]
Oct 31 21:51:24.941009 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.940985    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-efdee4-bf759994c-8q9gs"]
Oct 31 21:51:24.953680 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:24.953653    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-efdee4-bf759994c-8q9gs"]
Oct 31 21:51:26.225735 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:26.225706    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="05a4225d-a8de-4e82-9b4d-ff819a2c5603" path="/var/lib/kubelet/pods/05a4225d-a8de-4e82-9b4d-ff819a2c5603/volumes"
Oct 31 21:51:26.226207 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:26.226147    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="18cfce38-8c7b-45be-b7e9-cdb545b104ec" path="/var/lib/kubelet/pods/18cfce38-8c7b-45be-b7e9-cdb545b104ec/volumes"
Oct 31 21:51:26.406962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:26.401888    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:28.123121 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:28.123088    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:28.899637 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:28.899598    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:29.016668 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:29.016632    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:29.219521 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:29.219437    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"]
Oct 31 21:51:29.219994 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:29.219664    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp" podUID="e424a250-e02c-48f5-b215-a37f7de92b50" containerName="istio-proxy" containerID="cri-o://5d861c1b0b47dc16c28e8c4999955cfd9299694782ae9285fdcf8620964eed27" gracePeriod=30
Oct 31 21:51:29.245900 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:29.245865    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:29.815789 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:29.815757    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:29.831822 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:29.831794    4460 patch_prober.go:29] interesting pod/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 21:51:29.831995 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:29.831841    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp" podUID="e424a250-e02c-48f5-b215-a37f7de92b50" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 21:51:29.926133 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:29.926090    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:30.303319 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.303284    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"]
Oct 31 21:51:30.304168 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.304143    4460 cpu_manager.go:414] "containerMap: removing container" podUID="18cfce38-8c7b-45be-b7e9-cdb545b104ec" containerName="payload-processing"
Oct 31 21:51:30.304299 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.304181    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="18cfce38-8c7b-45be-b7e9-cdb545b104ec" containerName="payload-processing"
Oct 31 21:51:30.304299 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.304195    4460 cpu_manager.go:414] "containerMap: removing container" podUID="0d608736-5306-4c4a-9c55-fa8ee13af3ff" containerName="main"
Oct 31 21:51:30.304299 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.304201    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d608736-5306-4c4a-9c55-fa8ee13af3ff" containerName="main"
Oct 31 21:51:30.304299 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.304223    4460 cpu_manager.go:414] "containerMap: removing container" podUID="05a4225d-a8de-4e82-9b4d-ff819a2c5603" containerName="payload-pre-processing"
Oct 31 21:51:30.304299 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.304230    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="05a4225d-a8de-4e82-9b4d-ff819a2c5603" containerName="payload-pre-processing"
Oct 31 21:51:30.304461 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.304361    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="18cfce38-8c7b-45be-b7e9-cdb545b104ec" containerName="payload-processing"
Oct 31 21:51:30.304461 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.304368    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d608736-5306-4c4a-9c55-fa8ee13af3ff" containerName="main"
Oct 31 21:51:30.304461 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.304377    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="05a4225d-a8de-4e82-9b4d-ff819a2c5603" containerName="payload-pre-processing"
Oct 31 21:51:30.308618 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.308594    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:51:30.321311 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.321280    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-credential-socket\") pod \"e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") " pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:51:30.321550 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.321531    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-istio-data\") pod \"e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") " pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:51:30.321678 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.321662    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-workload-socket\") pod \"e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") " pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:51:30.321879 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.321859    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcwb5\" (UniqueName: \"kubernetes.io/projected/c6f14d22-d962-4176-bf59-afa4750db6c9-kube-api-access-mcwb5\") pod \"e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") " pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:51:30.322031 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.322012    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6f14d22-d962-4176-bf59-afa4750db6c9-istio-podinfo\") pod \"e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") " pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:51:30.322218 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.322200    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-istio-envoy\") pod \"e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") " pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:51:30.322323 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.322308    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6f14d22-d962-4176-bf59-afa4750db6c9-istiod-ca-cert\") pod \"e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") " pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:51:30.322441 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.322425    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-workload-certs\") pod \"e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") " pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:51:30.322601 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.322572    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6f14d22-d962-4176-bf59-afa4750db6c9-istio-token\") pod \"e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") " pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:51:30.335186 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.335161    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"]
Oct 31 21:51:30.343140 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.343119    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-d7fd1a-openshift-default-dockercfg-4ngt4\""
Oct 31 21:51:30.384189 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.384148    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:30.453990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.452102    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcwb5\" (UniqueName: \"kubernetes.io/projected/c6f14d22-d962-4176-bf59-afa4750db6c9-kube-api-access-mcwb5\") pod \"e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") " pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:51:30.453990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.452153    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6f14d22-d962-4176-bf59-afa4750db6c9-istio-podinfo\") pod \"e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") " pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:51:30.453990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.452258    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-istio-envoy\") pod \"e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") " pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:51:30.453990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.452282    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6f14d22-d962-4176-bf59-afa4750db6c9-istiod-ca-cert\") pod \"e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") " pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:51:30.453990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.452313    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-workload-certs\") pod \"e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") " pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:51:30.500048 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.474798    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6f14d22-d962-4176-bf59-afa4750db6c9-istio-token\") pod \"e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") " pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:51:30.500048 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.475060    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-credential-socket\") pod \"e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") " pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:51:30.500048 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.475283    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-istio-data\") pod \"e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") " pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:51:30.500048 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.475317    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-workload-socket\") pod \"e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") " pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:51:30.500048 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.475803    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-workload-socket\") pod \"e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") " pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:51:30.500048 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.478817    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-workload-certs\") pod \"e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") " pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:51:30.500048 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.479756    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-credential-socket\") pod \"e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") " pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:51:30.500048 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.480586    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-istio-data\") pod \"e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") " pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:51:30.500048 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.491673    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-istio-envoy\") pod \"e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") " pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:51:30.516975 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.508489    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6f14d22-d962-4176-bf59-afa4750db6c9-istiod-ca-cert\") pod \"e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") " pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:51:30.558443 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.548652    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6f14d22-d962-4176-bf59-afa4750db6c9-istio-podinfo\") pod \"e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") " pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:51:30.562521 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.562440    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcwb5\" (UniqueName: \"kubernetes.io/projected/c6f14d22-d962-4176-bf59-afa4750db6c9-kube-api-access-mcwb5\") pod \"e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") " pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:51:30.562892 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.562853    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6f14d22-d962-4176-bf59-afa4750db6c9-istio-token\") pod \"e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") " pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:51:30.591027 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.581878    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:30.668966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:30.631226    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:51:31.012188 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:31.012158    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"]
Oct 31 21:51:31.012934 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:51:31.012902    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6f14d22_d962_4176_bf59_afa4750db6c9.slice/crio-ee8bf159c14d62ed15bf495401ba242f6fa2e9ff65a552efbb8c8cbad16d4738 WatchSource:0}: Error finding container ee8bf159c14d62ed15bf495401ba242f6fa2e9ff65a552efbb8c8cbad16d4738: Status 404 returned error can't find the container with id ee8bf159c14d62ed15bf495401ba242f6fa2e9ff65a552efbb8c8cbad16d4738
Oct 31 21:51:31.015520 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:31.015488    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:51:31.015625 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:31.015586    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:51:31.015695 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:31.015635    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:51:31.701347 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:31.701314    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:31.894564 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:31.894527    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x" event={"ID":"c6f14d22-d962-4176-bf59-afa4750db6c9","Type":"ContainerStarted","Data":"30f0285c1cdffed585023d14c670568f83b6423881debea86a356a479304bdcb"}
Oct 31 21:51:31.894759 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:31.894578    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x" event={"ID":"c6f14d22-d962-4176-bf59-afa4750db6c9","Type":"ContainerStarted","Data":"ee8bf159c14d62ed15bf495401ba242f6fa2e9ff65a552efbb8c8cbad16d4738"}
Oct 31 21:51:31.957976 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:31.957858    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x" podStartSLOduration=1.957844986 podStartE2EDuration="1.957844986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:51:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:51:31.957023271 +0000 UTC m=+2131.873332699" watchObservedRunningTime="2026-10-31 21:51:31.957844986 +0000 UTC m=+2131.874154413"
Oct 31 21:51:32.322139 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:32.322102    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:32.444157 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:32.444116    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:32.632475 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:32.632393    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:51:32.637848 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:32.637822    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:51:32.649594 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:32.649563    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:32.901473 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:32.901383    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:51:32.902351 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:32.902332    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:51:33.529350 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:33.529311    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:34.505525 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.505464    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:51:34.552614 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.552584    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-credential-socket\") pod \"e424a250-e02c-48f5-b215-a37f7de92b50\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") "
Oct 31 21:51:34.552791 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.552683    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-workload-socket\") pod \"e424a250-e02c-48f5-b215-a37f7de92b50\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") "
Oct 31 21:51:34.552791 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.552739    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e424a250-e02c-48f5-b215-a37f7de92b50-kube-api-access-d4f5j\" (UniqueName: \"kubernetes.io/projected/e424a250-e02c-48f5-b215-a37f7de92b50-kube-api-access-d4f5j\") pod \"e424a250-e02c-48f5-b215-a37f7de92b50\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") "
Oct 31 21:51:34.552791 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.552763    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-istio-envoy\") pod \"e424a250-e02c-48f5-b215-a37f7de92b50\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") "
Oct 31 21:51:34.552991 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.552835    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-credential-socket" pod "e424a250-e02c-48f5-b215-a37f7de92b50" (UID: "e424a250-e02c-48f5-b215-a37f7de92b50"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:51:34.552991 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.552897    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-istio-data\") pod \"e424a250-e02c-48f5-b215-a37f7de92b50\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") "
Oct 31 21:51:34.553090 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.553036    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-workload-socket" pod "e424a250-e02c-48f5-b215-a37f7de92b50" (UID: "e424a250-e02c-48f5-b215-a37f7de92b50"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:51:34.553139 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.553080    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e424a250-e02c-48f5-b215-a37f7de92b50-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e424a250-e02c-48f5-b215-a37f7de92b50-istiod-ca-cert\") pod \"e424a250-e02c-48f5-b215-a37f7de92b50\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") "
Oct 31 21:51:34.553198 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.553136    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-istio-data" pod "e424a250-e02c-48f5-b215-a37f7de92b50" (UID: "e424a250-e02c-48f5-b215-a37f7de92b50"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:51:34.553198 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.553175    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-workload-certs\") pod \"e424a250-e02c-48f5-b215-a37f7de92b50\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") "
Oct 31 21:51:34.553299 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.553214    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e424a250-e02c-48f5-b215-a37f7de92b50-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e424a250-e02c-48f5-b215-a37f7de92b50-istio-podinfo\") pod \"e424a250-e02c-48f5-b215-a37f7de92b50\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") "
Oct 31 21:51:34.553299 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.553236    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e424a250-e02c-48f5-b215-a37f7de92b50-istio-token\" (UniqueName: \"kubernetes.io/projected/e424a250-e02c-48f5-b215-a37f7de92b50-istio-token\") pod \"e424a250-e02c-48f5-b215-a37f7de92b50\" (UID: \"e424a250-e02c-48f5-b215-a37f7de92b50\") "
Oct 31 21:51:34.553458 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.553421    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-workload-certs" pod "e424a250-e02c-48f5-b215-a37f7de92b50" (UID: "e424a250-e02c-48f5-b215-a37f7de92b50"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:51:34.553582 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.553471    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e424a250-e02c-48f5-b215-a37f7de92b50-istiod-ca-cert" pod "e424a250-e02c-48f5-b215-a37f7de92b50" (UID: "e424a250-e02c-48f5-b215-a37f7de92b50"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:51:34.553887 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.553867    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:51:34.554001 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.553892    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e424a250-e02c-48f5-b215-a37f7de92b50-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:51:34.554001 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.553907    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:51:34.554001 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.553919    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:51:34.554001 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.553933    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:51:34.555103 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.555076    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e424a250-e02c-48f5-b215-a37f7de92b50-kube-api-access-d4f5j" pod "e424a250-e02c-48f5-b215-a37f7de92b50" (UID: "e424a250-e02c-48f5-b215-a37f7de92b50"). InnerVolumeSpecName "kube-api-access-d4f5j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:51:34.555299 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.555280    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e424a250-e02c-48f5-b215-a37f7de92b50-istio-token" pod "e424a250-e02c-48f5-b215-a37f7de92b50" (UID: "e424a250-e02c-48f5-b215-a37f7de92b50"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:51:34.555364 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.555305    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-istio-envoy" pod "e424a250-e02c-48f5-b215-a37f7de92b50" (UID: "e424a250-e02c-48f5-b215-a37f7de92b50"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:51:34.555596 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.555579    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e424a250-e02c-48f5-b215-a37f7de92b50-istio-podinfo" pod "e424a250-e02c-48f5-b215-a37f7de92b50" (UID: "e424a250-e02c-48f5-b215-a37f7de92b50"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:51:34.654903 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.654870    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e424a250-e02c-48f5-b215-a37f7de92b50-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:51:34.654903 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.654897    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e424a250-e02c-48f5-b215-a37f7de92b50-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:51:34.654903 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.654906    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4f5j\" (UniqueName: \"kubernetes.io/projected/e424a250-e02c-48f5-b215-a37f7de92b50-kube-api-access-d4f5j\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:51:34.655148 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.654915    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e424a250-e02c-48f5-b215-a37f7de92b50-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:51:34.916437 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.916403    4460 generic.go:356] "Generic (PLEG): container finished" podID="e424a250-e02c-48f5-b215-a37f7de92b50" containerID="5d861c1b0b47dc16c28e8c4999955cfd9299694782ae9285fdcf8620964eed27" exitCode=0
Oct 31 21:51:34.916612 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.916453    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"
Oct 31 21:51:34.916612 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.916482    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp" event={"ID":"e424a250-e02c-48f5-b215-a37f7de92b50","Type":"ContainerDied","Data":"5d861c1b0b47dc16c28e8c4999955cfd9299694782ae9285fdcf8620964eed27"}
Oct 31 21:51:34.916612 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.916540    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp" event={"ID":"e424a250-e02c-48f5-b215-a37f7de92b50","Type":"ContainerDied","Data":"d1b2fae010c4b8f189a8083ddea51e079f5396a86cc57b1780284c32cdb5e265"}
Oct 31 21:51:34.916612 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.916571    4460 scope.go:122] "RemoveContainer" containerID="5d861c1b0b47dc16c28e8c4999955cfd9299694782ae9285fdcf8620964eed27"
Oct 31 21:51:34.927872 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.927850    4460 scope.go:122] "RemoveContainer" containerID="66b37cf79b2f6cef0b5478c18ba9244b61b49f14ba60f45e55decfb3746370a3"
Oct 31 21:51:34.937806 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.937789    4460 scope.go:122] "RemoveContainer" containerID="5d861c1b0b47dc16c28e8c4999955cfd9299694782ae9285fdcf8620964eed27"
Oct 31 21:51:34.938100 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:51:34.938080    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d861c1b0b47dc16c28e8c4999955cfd9299694782ae9285fdcf8620964eed27\": container with ID starting with 5d861c1b0b47dc16c28e8c4999955cfd9299694782ae9285fdcf8620964eed27 not found: ID does not exist" containerID="5d861c1b0b47dc16c28e8c4999955cfd9299694782ae9285fdcf8620964eed27"
Oct 31 21:51:34.938190 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.938110    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d861c1b0b47dc16c28e8c4999955cfd9299694782ae9285fdcf8620964eed27"} err="failed to get container status \"5d861c1b0b47dc16c28e8c4999955cfd9299694782ae9285fdcf8620964eed27\": rpc error: code = NotFound desc = could not find container \"5d861c1b0b47dc16c28e8c4999955cfd9299694782ae9285fdcf8620964eed27\": container with ID starting with 5d861c1b0b47dc16c28e8c4999955cfd9299694782ae9285fdcf8620964eed27 not found: ID does not exist"
Oct 31 21:51:34.938190 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.938130    4460 scope.go:122] "RemoveContainer" containerID="66b37cf79b2f6cef0b5478c18ba9244b61b49f14ba60f45e55decfb3746370a3"
Oct 31 21:51:34.938429 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:51:34.938408    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66b37cf79b2f6cef0b5478c18ba9244b61b49f14ba60f45e55decfb3746370a3\": container with ID starting with 66b37cf79b2f6cef0b5478c18ba9244b61b49f14ba60f45e55decfb3746370a3 not found: ID does not exist" containerID="66b37cf79b2f6cef0b5478c18ba9244b61b49f14ba60f45e55decfb3746370a3"
Oct 31 21:51:34.938469 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.938436    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66b37cf79b2f6cef0b5478c18ba9244b61b49f14ba60f45e55decfb3746370a3"} err="failed to get container status \"66b37cf79b2f6cef0b5478c18ba9244b61b49f14ba60f45e55decfb3746370a3\": rpc error: code = NotFound desc = could not find container \"66b37cf79b2f6cef0b5478c18ba9244b61b49f14ba60f45e55decfb3746370a3\": container with ID starting with 66b37cf79b2f6cef0b5478c18ba9244b61b49f14ba60f45e55decfb3746370a3 not found: ID does not exist"
Oct 31 21:51:34.976278 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.976249    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"]
Oct 31 21:51:34.990445 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:34.990417    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-efdee4-openshift-default-778dd4f665-dxlxp"]
Oct 31 21:51:35.245530 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:35.245430    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:36.226490 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.226456    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e424a250-e02c-48f5-b215-a37f7de92b50" path="/var/lib/kubelet/pods/e424a250-e02c-48f5-b215-a37f7de92b50/volumes"
Oct 31 21:51:36.650255 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.650221    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk"]
Oct 31 21:51:36.650875 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.650856    4460 cpu_manager.go:414] "containerMap: removing container" podUID="e424a250-e02c-48f5-b215-a37f7de92b50" containerName="istio-proxy"
Oct 31 21:51:36.650875 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.650872    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e424a250-e02c-48f5-b215-a37f7de92b50" containerName="istio-proxy"
Oct 31 21:51:36.651090 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.650896    4460 cpu_manager.go:414] "containerMap: removing container" podUID="e424a250-e02c-48f5-b215-a37f7de92b50" containerName="istio-proxy"
Oct 31 21:51:36.651090 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.650902    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e424a250-e02c-48f5-b215-a37f7de92b50" containerName="istio-proxy"
Oct 31 21:51:36.651090 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.651023    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="e424a250-e02c-48f5-b215-a37f7de92b50" containerName="istio-proxy"
Oct 31 21:51:36.651090 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.651036    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="e424a250-e02c-48f5-b215-a37f7de92b50" containerName="istio-proxy"
Oct 31 21:51:36.656596 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.656576    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk"
Oct 31 21:51:36.676010 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.675984    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk"]
Oct 31 21:51:36.684477 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.684454    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-d7fd1a\"/\"kube-root-ca.crt\""
Oct 31 21:51:36.689187 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.689165    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-d7fd1a\"/\"default-dockercfg-42zcn\""
Oct 31 21:51:36.689293 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.689222    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-d7fd1a\"/\"openshift-service-ca.crt\""
Oct 31 21:51:36.689293 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.689253    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-d7fd1a\"/\"noten-model-d7fd1a-kserve-self-signed-certs\""
Oct 31 21:51:36.720526 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.720494    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:36.782337 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.782298    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-tls-certs\") pod \"noten-model-d7fd1a-kserve-558c56d88f-ngmbk\" (UID: \"0aa2e7f9-730b-43a1-af62-08bd454f7cf5\") " pod="ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk"
Oct 31 21:51:36.782527 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.782451    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-model-cache\") pod \"noten-model-d7fd1a-kserve-558c56d88f-ngmbk\" (UID: \"0aa2e7f9-730b-43a1-af62-08bd454f7cf5\") " pod="ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk"
Oct 31 21:51:36.782527 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.782515    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5l7r\" (UniqueName: \"kubernetes.io/projected/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-kube-api-access-p5l7r\") pod \"noten-model-d7fd1a-kserve-558c56d88f-ngmbk\" (UID: \"0aa2e7f9-730b-43a1-af62-08bd454f7cf5\") " pod="ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk"
Oct 31 21:51:36.782621 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.782601    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-dshm\") pod \"noten-model-d7fd1a-kserve-558c56d88f-ngmbk\" (UID: \"0aa2e7f9-730b-43a1-af62-08bd454f7cf5\") " pod="ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk"
Oct 31 21:51:36.782717 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.782622    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-tmp-dir\") pod \"noten-model-d7fd1a-kserve-558c56d88f-ngmbk\" (UID: \"0aa2e7f9-730b-43a1-af62-08bd454f7cf5\") " pod="ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk"
Oct 31 21:51:36.782767 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.782747    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-home\") pod \"noten-model-d7fd1a-kserve-558c56d88f-ngmbk\" (UID: \"0aa2e7f9-730b-43a1-af62-08bd454f7cf5\") " pod="ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk"
Oct 31 21:51:36.883613 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.883577    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-tls-certs\") pod \"noten-model-d7fd1a-kserve-558c56d88f-ngmbk\" (UID: \"0aa2e7f9-730b-43a1-af62-08bd454f7cf5\") " pod="ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk"
Oct 31 21:51:36.883788 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.883658    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-model-cache\") pod \"noten-model-d7fd1a-kserve-558c56d88f-ngmbk\" (UID: \"0aa2e7f9-730b-43a1-af62-08bd454f7cf5\") " pod="ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk"
Oct 31 21:51:36.883788 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.883679    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5l7r\" (UniqueName: \"kubernetes.io/projected/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-kube-api-access-p5l7r\") pod \"noten-model-d7fd1a-kserve-558c56d88f-ngmbk\" (UID: \"0aa2e7f9-730b-43a1-af62-08bd454f7cf5\") " pod="ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk"
Oct 31 21:51:36.883788 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.883731    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-dshm\") pod \"noten-model-d7fd1a-kserve-558c56d88f-ngmbk\" (UID: \"0aa2e7f9-730b-43a1-af62-08bd454f7cf5\") " pod="ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk"
Oct 31 21:51:36.883788 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.883753    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-tmp-dir\") pod \"noten-model-d7fd1a-kserve-558c56d88f-ngmbk\" (UID: \"0aa2e7f9-730b-43a1-af62-08bd454f7cf5\") " pod="ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk"
Oct 31 21:51:36.884030 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.883843    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-home\") pod \"noten-model-d7fd1a-kserve-558c56d88f-ngmbk\" (UID: \"0aa2e7f9-730b-43a1-af62-08bd454f7cf5\") " pod="ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk"
Oct 31 21:51:36.884229 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.884199    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-model-cache\") pod \"noten-model-d7fd1a-kserve-558c56d88f-ngmbk\" (UID: \"0aa2e7f9-730b-43a1-af62-08bd454f7cf5\") " pod="ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk"
Oct 31 21:51:36.884322 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.884281    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-tmp-dir\") pod \"noten-model-d7fd1a-kserve-558c56d88f-ngmbk\" (UID: \"0aa2e7f9-730b-43a1-af62-08bd454f7cf5\") " pod="ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk"
Oct 31 21:51:36.884322 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.884293    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-home\") pod \"noten-model-d7fd1a-kserve-558c56d88f-ngmbk\" (UID: \"0aa2e7f9-730b-43a1-af62-08bd454f7cf5\") " pod="ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk"
Oct 31 21:51:36.885996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.885974    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-dshm\") pod \"noten-model-d7fd1a-kserve-558c56d88f-ngmbk\" (UID: \"0aa2e7f9-730b-43a1-af62-08bd454f7cf5\") " pod="ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk"
Oct 31 21:51:36.886219 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.886200    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-tls-certs\") pod \"noten-model-d7fd1a-kserve-558c56d88f-ngmbk\" (UID: \"0aa2e7f9-730b-43a1-af62-08bd454f7cf5\") " pod="ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk"
Oct 31 21:51:36.914340 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.914284    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5l7r\" (UniqueName: \"kubernetes.io/projected/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-kube-api-access-p5l7r\") pod \"noten-model-d7fd1a-kserve-558c56d88f-ngmbk\" (UID: \"0aa2e7f9-730b-43a1-af62-08bd454f7cf5\") " pod="ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk"
Oct 31 21:51:36.986857 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:36.986557    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk"
Oct 31 21:51:37.170823 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:37.170483    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:37.376970 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:37.376928    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk"]
Oct 31 21:51:37.377814 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:51:37.377787    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0aa2e7f9_730b_43a1_af62_08bd454f7cf5.slice/crio-55d9077e9f833379ddc1992224a06687697473a45435b31a897be19592177a17 WatchSource:0}: Error finding container 55d9077e9f833379ddc1992224a06687697473a45435b31a897be19592177a17: Status 404 returned error can't find the container with id 55d9077e9f833379ddc1992224a06687697473a45435b31a897be19592177a17
Oct 31 21:51:38.018306 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:38.018270    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk" event={"ID":"0aa2e7f9-730b-43a1-af62-08bd454f7cf5","Type":"ContainerStarted","Data":"a50b3cc47e075e190ad8fc5904c15f997717f06de4e0616e4fdd071263305b77"}
Oct 31 21:51:38.018484 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:38.018314    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk" event={"ID":"0aa2e7f9-730b-43a1-af62-08bd454f7cf5","Type":"ContainerStarted","Data":"55d9077e9f833379ddc1992224a06687697473a45435b31a897be19592177a17"}
Oct 31 21:51:38.068805 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:38.068751    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk" podStartSLOduration=2.068737399 podStartE2EDuration="2.068737399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:51:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:51:38.066923601 +0000 UTC m=+2137.983233033" watchObservedRunningTime="2026-10-31 21:51:38.068737399 +0000 UTC m=+2137.985046827"
Oct 31 21:51:38.988114 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:38.988077    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk"
Oct 31 21:51:39.040287 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:39.040252    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:40.825341 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:40.825308    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:42.513273 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:42.513241    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:44.207715 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:44.207677    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:45.992702 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:45.992662    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:46.987525 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:46.987488    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk"
Oct 31 21:51:46.999192 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:46.999170    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk"
Oct 31 21:51:47.090171 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:47.090146    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk"
Oct 31 21:51:47.733474 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:47.733440    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:47.920709 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:47.920675    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:48.382702 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:48.382668    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:51:49.548588 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:49.548553    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:51.285125 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:51.285085    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:51.452595 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:51:51.452561    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:51:53.107794 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:53.107755    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:54.755880 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:54.755842    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:55.575432 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:55.575386    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:56.406469 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:56.406430    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:58.138792 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:58.138756    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:59.826791 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:59.826756    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:51:59.938352 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:59.938317    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk"]
Oct 31 21:51:59.953138 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:51:59.953101    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:00.138927 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:00.137553    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:00.163191 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:00.163149    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"]
Oct 31 21:52:00.163713 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:00.163663    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x" podUID="c6f14d22-d962-4176-bf59-afa4750db6c9" containerName="istio-proxy" containerID="cri-o://30f0285c1cdffed585023d14c670568f83b6423881debea86a356a479304bdcb" gracePeriod=30
Oct 31 21:52:00.454545 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:00.454460    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:00.607644 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:00.607607    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:01.635633 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:01.635596    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:02.903683 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:02.903651    4460 patch_prober.go:29] interesting pod/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Oct 31 21:52:02.904085 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:02.903714    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x" podUID="c6f14d22-d962-4176-bf59-afa4750db6c9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 21:52:03.283294 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:03.283255    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:05.124226 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.124176    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:05.392905 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.392844    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:52:05.445685 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.445654    4460 generic.go:356] "Generic (PLEG): container finished" podID="c6f14d22-d962-4176-bf59-afa4750db6c9" containerID="30f0285c1cdffed585023d14c670568f83b6423881debea86a356a479304bdcb" exitCode=0
Oct 31 21:52:05.445815 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.445701    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"
Oct 31 21:52:05.445815 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.445719    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x" event={"ID":"c6f14d22-d962-4176-bf59-afa4750db6c9","Type":"ContainerDied","Data":"30f0285c1cdffed585023d14c670568f83b6423881debea86a356a479304bdcb"}
Oct 31 21:52:05.445815 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.445797    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x" event={"ID":"c6f14d22-d962-4176-bf59-afa4750db6c9","Type":"ContainerDied","Data":"ee8bf159c14d62ed15bf495401ba242f6fa2e9ff65a552efbb8c8cbad16d4738"}
Oct 31 21:52:05.445916 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.445825    4460 scope.go:122] "RemoveContainer" containerID="30f0285c1cdffed585023d14c670568f83b6423881debea86a356a479304bdcb"
Oct 31 21:52:05.455824 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.455799    4460 scope.go:122] "RemoveContainer" containerID="30f0285c1cdffed585023d14c670568f83b6423881debea86a356a479304bdcb"
Oct 31 21:52:05.456204 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:52:05.456176    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30f0285c1cdffed585023d14c670568f83b6423881debea86a356a479304bdcb\": container with ID starting with 30f0285c1cdffed585023d14c670568f83b6423881debea86a356a479304bdcb not found: ID does not exist" containerID="30f0285c1cdffed585023d14c670568f83b6423881debea86a356a479304bdcb"
Oct 31 21:52:05.456290 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.456204    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30f0285c1cdffed585023d14c670568f83b6423881debea86a356a479304bdcb"} err="failed to get container status \"30f0285c1cdffed585023d14c670568f83b6423881debea86a356a479304bdcb\": rpc error: code = NotFound desc = could not find container \"30f0285c1cdffed585023d14c670568f83b6423881debea86a356a479304bdcb\": container with ID starting with 30f0285c1cdffed585023d14c670568f83b6423881debea86a356a479304bdcb not found: ID does not exist"
Oct 31 21:52:05.475059 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.475032    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-workload-certs\") pod \"c6f14d22-d962-4176-bf59-afa4750db6c9\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") "
Oct 31 21:52:05.475200 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.475081    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-credential-socket\") pod \"c6f14d22-d962-4176-bf59-afa4750db6c9\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") "
Oct 31 21:52:05.475200 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.475146    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-istio-data\") pod \"c6f14d22-d962-4176-bf59-afa4750db6c9\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") "
Oct 31 21:52:05.475323 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.475234    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-istio-envoy\") pod \"c6f14d22-d962-4176-bf59-afa4750db6c9\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") "
Oct 31 21:52:05.475387 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.475364    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-workload-certs" pod "c6f14d22-d962-4176-bf59-afa4750db6c9" (UID: "c6f14d22-d962-4176-bf59-afa4750db6c9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:52:05.475444 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.475418    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c6f14d22-d962-4176-bf59-afa4750db6c9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6f14d22-d962-4176-bf59-afa4750db6c9-istio-podinfo\") pod \"c6f14d22-d962-4176-bf59-afa4750db6c9\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") "
Oct 31 21:52:05.476714 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.475497    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-workload-socket\") pod \"c6f14d22-d962-4176-bf59-afa4750db6c9\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") "
Oct 31 21:52:05.476714 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.475619    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c6f14d22-d962-4176-bf59-afa4750db6c9-kube-api-access-mcwb5\" (UniqueName: \"kubernetes.io/projected/c6f14d22-d962-4176-bf59-afa4750db6c9-kube-api-access-mcwb5\") pod \"c6f14d22-d962-4176-bf59-afa4750db6c9\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") "
Oct 31 21:52:05.476714 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.475655    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-credential-socket" pod "c6f14d22-d962-4176-bf59-afa4750db6c9" (UID: "c6f14d22-d962-4176-bf59-afa4750db6c9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:52:05.476714 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.475676    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c6f14d22-d962-4176-bf59-afa4750db6c9-istio-token\" (UniqueName: \"kubernetes.io/projected/c6f14d22-d962-4176-bf59-afa4750db6c9-istio-token\") pod \"c6f14d22-d962-4176-bf59-afa4750db6c9\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") "
Oct 31 21:52:05.476714 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.475704    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c6f14d22-d962-4176-bf59-afa4750db6c9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6f14d22-d962-4176-bf59-afa4750db6c9-istiod-ca-cert\") pod \"c6f14d22-d962-4176-bf59-afa4750db6c9\" (UID: \"c6f14d22-d962-4176-bf59-afa4750db6c9\") "
Oct 31 21:52:05.476714 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.476312    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:05.476714 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.476333    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:05.476714 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.476524    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-istio-data" pod "c6f14d22-d962-4176-bf59-afa4750db6c9" (UID: "c6f14d22-d962-4176-bf59-afa4750db6c9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:52:05.476714 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.476664    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6f14d22-d962-4176-bf59-afa4750db6c9-istiod-ca-cert" pod "c6f14d22-d962-4176-bf59-afa4750db6c9" (UID: "c6f14d22-d962-4176-bf59-afa4750db6c9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:52:05.477387 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.477201    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-workload-socket" pod "c6f14d22-d962-4176-bf59-afa4750db6c9" (UID: "c6f14d22-d962-4176-bf59-afa4750db6c9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:52:05.483361 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.483132    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-istio-envoy" pod "c6f14d22-d962-4176-bf59-afa4750db6c9" (UID: "c6f14d22-d962-4176-bf59-afa4750db6c9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:52:05.483361 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.483153    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c6f14d22-d962-4176-bf59-afa4750db6c9-istio-podinfo" pod "c6f14d22-d962-4176-bf59-afa4750db6c9" (UID: "c6f14d22-d962-4176-bf59-afa4750db6c9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:52:05.483361 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.483259    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6f14d22-d962-4176-bf59-afa4750db6c9-istio-token" pod "c6f14d22-d962-4176-bf59-afa4750db6c9" (UID: "c6f14d22-d962-4176-bf59-afa4750db6c9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:52:05.483361 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.483331    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6f14d22-d962-4176-bf59-afa4750db6c9-kube-api-access-mcwb5" pod "c6f14d22-d962-4176-bf59-afa4750db6c9" (UID: "c6f14d22-d962-4176-bf59-afa4750db6c9"). InnerVolumeSpecName "kube-api-access-mcwb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:52:05.577497 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.577468    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6f14d22-d962-4176-bf59-afa4750db6c9-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:05.577497 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.577496    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:05.577651 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.577505    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcwb5\" (UniqueName: \"kubernetes.io/projected/c6f14d22-d962-4176-bf59-afa4750db6c9-kube-api-access-mcwb5\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:05.577651 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.577515    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6f14d22-d962-4176-bf59-afa4750db6c9-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:05.577651 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.577522    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6f14d22-d962-4176-bf59-afa4750db6c9-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:05.577651 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.577530    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:05.577651 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.577537    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6f14d22-d962-4176-bf59-afa4750db6c9-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:05.765342 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.765308    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:05.797857 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.797829    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"]
Oct 31 21:52:05.811426 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:05.811401    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-d7fd1a-openshift-default-77f6fb6659-4mx2x"]
Oct 31 21:52:06.227060 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:06.227029    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c6f14d22-d962-4176-bf59-afa4750db6c9" path="/var/lib/kubelet/pods/c6f14d22-d962-4176-bf59-afa4750db6c9/volumes"
Oct 31 21:52:06.855041 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:06.855002    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:08.310075 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:08.310043    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:08.550425 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:08.550390    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:10.324732 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:10.324697    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:12.121301 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:12.121265    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:13.744428 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:13.744391    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:14.057863 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:14.057818    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:14.893188 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:14.893156    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:14.966836 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:14.966801    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk" podUID="0aa2e7f9-730b-43a1-af62-08bd454f7cf5" containerName="main" containerID="cri-o://a50b3cc47e075e190ad8fc5904c15f997717f06de4e0616e4fdd071263305b77" gracePeriod=45
Oct 31 21:52:15.080774 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:15.080740    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk" podUID="0aa2e7f9-730b-43a1-af62-08bd454f7cf5" containerName="main" probeResult="failure" output="Get \"https://10.133.0.156:8000/ready\": dial tcp 10.133.0.156:8000: connect: connection refused"
Oct 31 21:52:15.231226 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:15.231201    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk"
Oct 31 21:52:15.355395 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:15.355358    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-dshm\" (UniqueName: \"kubernetes.io/empty-dir/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-dshm\") pod \"0aa2e7f9-730b-43a1-af62-08bd454f7cf5\" (UID: \"0aa2e7f9-730b-43a1-af62-08bd454f7cf5\") "
Oct 31 21:52:15.355606 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:15.355407    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-home\" (UniqueName: \"kubernetes.io/empty-dir/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-home\") pod \"0aa2e7f9-730b-43a1-af62-08bd454f7cf5\" (UID: \"0aa2e7f9-730b-43a1-af62-08bd454f7cf5\") "
Oct 31 21:52:15.355606 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:15.355460    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-tls-certs\" (UniqueName: \"kubernetes.io/secret/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-tls-certs\") pod \"0aa2e7f9-730b-43a1-af62-08bd454f7cf5\" (UID: \"0aa2e7f9-730b-43a1-af62-08bd454f7cf5\") "
Oct 31 21:52:15.355740 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:15.355654    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-home" pod "0aa2e7f9-730b-43a1-af62-08bd454f7cf5" (UID: "0aa2e7f9-730b-43a1-af62-08bd454f7cf5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:52:15.355740 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:15.355685    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-model-cache\") pod \"0aa2e7f9-730b-43a1-af62-08bd454f7cf5\" (UID: \"0aa2e7f9-730b-43a1-af62-08bd454f7cf5\") "
Oct 31 21:52:15.355740 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:15.355719    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-kube-api-access-p5l7r\" (UniqueName: \"kubernetes.io/projected/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-kube-api-access-p5l7r\") pod \"0aa2e7f9-730b-43a1-af62-08bd454f7cf5\" (UID: \"0aa2e7f9-730b-43a1-af62-08bd454f7cf5\") "
Oct 31 21:52:15.355999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:15.355750    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-tmp-dir\") pod \"0aa2e7f9-730b-43a1-af62-08bd454f7cf5\" (UID: \"0aa2e7f9-730b-43a1-af62-08bd454f7cf5\") "
Oct 31 21:52:15.355999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:15.355870    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-model-cache" pod "0aa2e7f9-730b-43a1-af62-08bd454f7cf5" (UID: "0aa2e7f9-730b-43a1-af62-08bd454f7cf5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:52:15.356128 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:15.356090    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-tmp-dir" pod "0aa2e7f9-730b-43a1-af62-08bd454f7cf5" (UID: "0aa2e7f9-730b-43a1-af62-08bd454f7cf5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:52:15.356387 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:15.356367    4460 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-model-cache\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:15.356464 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:15.356391    4460 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-tmp-dir\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:15.356464 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:15.356404    4460 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-home\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:15.358253 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:15.358223    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-dshm" pod "0aa2e7f9-730b-43a1-af62-08bd454f7cf5" (UID: "0aa2e7f9-730b-43a1-af62-08bd454f7cf5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:52:15.358372 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:15.358272    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-tls-certs" pod "0aa2e7f9-730b-43a1-af62-08bd454f7cf5" (UID: "0aa2e7f9-730b-43a1-af62-08bd454f7cf5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 21:52:15.358372 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:15.358296    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-kube-api-access-p5l7r" pod "0aa2e7f9-730b-43a1-af62-08bd454f7cf5" (UID: "0aa2e7f9-730b-43a1-af62-08bd454f7cf5"). InnerVolumeSpecName "kube-api-access-p5l7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:52:15.457676 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:15.457592    4460 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-dshm\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:15.457676 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:15.457622    4460 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-tls-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:15.457676 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:15.457638    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5l7r\" (UniqueName: \"kubernetes.io/projected/0aa2e7f9-730b-43a1-af62-08bd454f7cf5-kube-api-access-p5l7r\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:15.518934 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:15.518899    4460 generic.go:356] "Generic (PLEG): container finished" podID="0aa2e7f9-730b-43a1-af62-08bd454f7cf5" containerID="a50b3cc47e075e190ad8fc5904c15f997717f06de4e0616e4fdd071263305b77" exitCode=0
Oct 31 21:52:15.519134 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:15.518974    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk" event={"ID":"0aa2e7f9-730b-43a1-af62-08bd454f7cf5","Type":"ContainerDied","Data":"a50b3cc47e075e190ad8fc5904c15f997717f06de4e0616e4fdd071263305b77"}
Oct 31 21:52:15.519134 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:15.518978    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk"
Oct 31 21:52:15.519134 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:15.519006    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk" event={"ID":"0aa2e7f9-730b-43a1-af62-08bd454f7cf5","Type":"ContainerDied","Data":"55d9077e9f833379ddc1992224a06687697473a45435b31a897be19592177a17"}
Oct 31 21:52:15.519134 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:15.519022    4460 scope.go:122] "RemoveContainer" containerID="a50b3cc47e075e190ad8fc5904c15f997717f06de4e0616e4fdd071263305b77"
Oct 31 21:52:15.529881 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:15.529861    4460 scope.go:122] "RemoveContainer" containerID="a50b3cc47e075e190ad8fc5904c15f997717f06de4e0616e4fdd071263305b77"
Oct 31 21:52:15.530252 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:52:15.530229    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a50b3cc47e075e190ad8fc5904c15f997717f06de4e0616e4fdd071263305b77\": container with ID starting with a50b3cc47e075e190ad8fc5904c15f997717f06de4e0616e4fdd071263305b77 not found: ID does not exist" containerID="a50b3cc47e075e190ad8fc5904c15f997717f06de4e0616e4fdd071263305b77"
Oct 31 21:52:15.530336 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:15.530261    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a50b3cc47e075e190ad8fc5904c15f997717f06de4e0616e4fdd071263305b77"} err="failed to get container status \"a50b3cc47e075e190ad8fc5904c15f997717f06de4e0616e4fdd071263305b77\": rpc error: code = NotFound desc = could not find container \"a50b3cc47e075e190ad8fc5904c15f997717f06de4e0616e4fdd071263305b77\": container with ID starting with a50b3cc47e075e190ad8fc5904c15f997717f06de4e0616e4fdd071263305b77 not found: ID does not exist"
Oct 31 21:52:15.580252 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:15.580228    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk"]
Oct 31 21:52:15.582221 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:15.582193    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:15.583451 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:15.583434    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-d7fd1a/noten-model-d7fd1a-kserve-558c56d88f-ngmbk"]
Oct 31 21:52:16.225448 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:16.225405    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0aa2e7f9-730b-43a1-af62-08bd454f7cf5" path="/var/lib/kubelet/pods/0aa2e7f9-730b-43a1-af62-08bd454f7cf5/volumes"
Oct 31 21:52:17.343296 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:17.343263    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:19.161366 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:19.161305    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:20.906394 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:20.906355    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:21.459401 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:52:21.459363    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:52:22.614381 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:22.614347    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:24.457586 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:24.457549    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:26.230646 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:26.230614    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:27.018628 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.018593    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:27.431292 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.431208    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"]
Oct 31 21:52:27.432212 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.432187    4460 cpu_manager.go:414] "containerMap: removing container" podUID="c6f14d22-d962-4176-bf59-afa4750db6c9" containerName="istio-proxy"
Oct 31 21:52:27.432282 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.432215    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c6f14d22-d962-4176-bf59-afa4750db6c9" containerName="istio-proxy"
Oct 31 21:52:27.432322 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.432293    4460 cpu_manager.go:414] "containerMap: removing container" podUID="0aa2e7f9-730b-43a1-af62-08bd454f7cf5" containerName="main"
Oct 31 21:52:27.432322 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.432302    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0aa2e7f9-730b-43a1-af62-08bd454f7cf5" containerName="main"
Oct 31 21:52:27.432488 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.432472    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="0aa2e7f9-730b-43a1-af62-08bd454f7cf5" containerName="main"
Oct 31 21:52:27.432534 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.432502    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="c6f14d22-d962-4176-bf59-afa4750db6c9" containerName="istio-proxy"
Oct 31 21:52:27.436569 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.436542    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:52:27.468655 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.468606    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-4bd51b-openshift-default-dockercfg-dncjx\""
Oct 31 21:52:27.468791 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.468689    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-workload-certs\") pod \"e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") " pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:52:27.468791 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.468758    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df2pd\" (UniqueName: \"kubernetes.io/projected/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-kube-api-access-df2pd\") pod \"e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") " pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:52:27.468908 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.468820    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istiod-ca-cert\") pod \"e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") " pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:52:27.468979 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.468954    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istio-token\") pod \"e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") " pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:52:27.469074 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.469057    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istio-data\") pod \"e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") " pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:52:27.469194 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.469098    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-workload-socket\") pod \"e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") " pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:52:27.469194 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.469167    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istio-envoy\") pod \"e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") " pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:52:27.469326 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.469209    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istio-podinfo\") pod \"e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") " pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:52:27.469326 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.469247    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-credential-socket\") pod \"e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") " pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:52:27.475570 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.475542    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"]
Oct 31 21:52:27.487089 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.487051    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:27.570517 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.570485    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-df2pd\" (UniqueName: \"kubernetes.io/projected/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-kube-api-access-df2pd\") pod \"e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") " pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:52:27.570660 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.570545    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istiod-ca-cert\") pod \"e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") " pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:52:27.570660 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.570593    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istio-token\") pod \"e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") " pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:52:27.570788 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.570662    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istio-data\") pod \"e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") " pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:52:27.570788 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.570684    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-workload-socket\") pod \"e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") " pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:52:27.570921 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.570891    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istio-envoy\") pod \"e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") " pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:52:27.571030 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.571004    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istio-podinfo\") pod \"e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") " pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:52:27.571092 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.571053    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-credential-socket\") pod \"e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") " pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:52:27.571149 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.571126    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istio-data\") pod \"e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") " pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:52:27.571215 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.571190    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-workload-socket\") pod \"e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") " pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:52:27.571215 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.571192    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-workload-certs\") pod \"e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") " pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:52:27.571437 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.571416    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istiod-ca-cert\") pod \"e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") " pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:52:27.571499 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.571432    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-credential-socket\") pod \"e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") " pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:52:27.571499 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.571456    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-workload-certs\") pod \"e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") " pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:52:27.573186 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.573167    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istio-envoy\") pod \"e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") " pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:52:27.573504 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.573485    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istio-podinfo\") pod \"e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") " pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:52:27.598372 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.598350    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istio-token\") pod \"e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") " pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:52:27.598572 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.598551    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-df2pd\" (UniqueName: \"kubernetes.io/projected/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-kube-api-access-df2pd\") pod \"e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") " pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:52:27.599670 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.599632    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:27.767015 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:27.765068    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:52:28.104484 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:28.104407    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:28.190757 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:28.190722    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"]
Oct 31 21:52:28.203872 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:28.203832    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:52:28.204122 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:28.204092    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:52:28.204235 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:28.204147    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:52:28.704796 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:28.704757    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8" event={"ID":"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3","Type":"ContainerStarted","Data":"76cf8b03daa7ffbdbfec18e344a67a0b3659024fdafdcd4780bc651790b61d1d"}
Oct 31 21:52:28.705305 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:28.704824    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8" event={"ID":"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3","Type":"ContainerStarted","Data":"f1c5127b2f628179f01f792868624f1372f02e9db0f0f78b4af62092d969e618"}
Oct 31 21:52:28.766024 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:28.765998    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:52:28.788919 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:28.788865    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8" podStartSLOduration=1.788852231 podStartE2EDuration="1.788852231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:52:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:52:28.7741519 +0000 UTC m=+2188.690461331" watchObservedRunningTime="2026-10-31 21:52:28.788852231 +0000 UTC m=+2188.705161658"
Oct 31 21:52:29.771699 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:29.771670    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:52:29.810287 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:29.810255    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:30.717860 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:30.717824    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:52:30.718804 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:30.718783    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:52:31.122262 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:31.122224    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:31.241400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:31.241369    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:31.737005 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:31.736966    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:32.822012 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:32.821972    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"]
Oct 31 21:52:32.822475 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:32.822276    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm" podUID="56d5066a-9abc-4b3d-b905-2250ef807321" containerName="istio-proxy" containerID="cri-o://c0b783b082ea28c91ea5400013bbbb4ca732339f5eee490598fd577660a3b4d5" gracePeriod=30
Oct 31 21:52:32.835596 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:32.835571    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"]
Oct 31 21:52:32.835810 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:32.835783    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7" podUID="b06d985f-9a38-4ea8-a120-676d0c3de8aa" containerName="istio-proxy" containerID="cri-o://47718e5e2043427be3b2865aab093df5294356bbbb85f14b0aa7551f379ea9f0" gracePeriod=30
Oct 31 21:52:32.836064 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:32.836032    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:32.846095 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:32.845451    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"]
Oct 31 21:52:32.846485 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:32.846449    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm" podUID="3996ffdd-3dbe-41f6-ab61-62102a86e76b" containerName="istio-proxy" containerID="cri-o://860bf94ed996535b0710e5576dc042d5c8eb988d8fcd7ad646f2c39412a662d6" gracePeriod=30
Oct 31 21:52:32.850407 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:32.850380    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"]
Oct 31 21:52:32.850735 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:32.850705    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g" podUID="96a8801d-ad9e-4268-8dd0-a657e235e83c" containerName="istio-proxy" containerID="cri-o://0c63ce52d24a8104077be1e4117001078753b5160412fecf19fc55adde974036" gracePeriod=30
Oct 31 21:52:32.961875 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:32.961842    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:32.995842 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:32.995806    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-4bd51b-6b775cddf9-ld82r"]
Oct 31 21:52:33.000545 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:33.000520    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-4bd51b-6b775cddf9-ld82r"
Oct 31 21:52:33.018741 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:33.018716    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-4bd51b\""
Oct 31 21:52:33.018870 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:33.018801    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-4bd51b-dockercfg-jflk4\""
Oct 31 21:52:33.022272 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:33.022250    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-4bd51b-6b775cddf9-ld82r"]
Oct 31 21:52:33.157020 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:33.122813    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d141422b-120e-4dc4-bfe0-95ab1391d419-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-4bd51b-6b775cddf9-ld82r\" (UID: \"d141422b-120e-4dc4-bfe0-95ab1391d419\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-4bd51b-6b775cddf9-ld82r"
Oct 31 21:52:33.158128 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:33.158087    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76v79\" (UniqueName: \"kubernetes.io/projected/d141422b-120e-4dc4-bfe0-95ab1391d419-kube-api-access-76v79\") pod \"payload-pre-processing-e2e-extmod-a-4bd51b-6b775cddf9-ld82r\" (UID: \"d141422b-120e-4dc4-bfe0-95ab1391d419\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-4bd51b-6b775cddf9-ld82r"
Oct 31 21:52:33.202009 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:33.195008    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-4bd51b-7d76c5559f-r978k"]
Oct 31 21:52:33.286962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:33.272626    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d141422b-120e-4dc4-bfe0-95ab1391d419-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-4bd51b-6b775cddf9-ld82r\" (UID: \"d141422b-120e-4dc4-bfe0-95ab1391d419\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-4bd51b-6b775cddf9-ld82r"
Oct 31 21:52:33.286962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:33.272759    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76v79\" (UniqueName: \"kubernetes.io/projected/d141422b-120e-4dc4-bfe0-95ab1391d419-kube-api-access-76v79\") pod \"payload-pre-processing-e2e-extmod-a-4bd51b-6b775cddf9-ld82r\" (UID: \"d141422b-120e-4dc4-bfe0-95ab1391d419\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-4bd51b-6b775cddf9-ld82r"
Oct 31 21:52:33.307739 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:33.307682    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d141422b-120e-4dc4-bfe0-95ab1391d419-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-4bd51b-6b775cddf9-ld82r\" (UID: \"d141422b-120e-4dc4-bfe0-95ab1391d419\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-4bd51b-6b775cddf9-ld82r"
Oct 31 21:52:33.318245 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:33.318219    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-4bd51b-7d76c5559f-r978k"]
Oct 31 21:52:33.318401 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:33.318330    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-4bd51b-7d76c5559f-r978k"
Oct 31 21:52:33.319499 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:33.319472    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76v79\" (UniqueName: \"kubernetes.io/projected/d141422b-120e-4dc4-bfe0-95ab1391d419-kube-api-access-76v79\") pod \"payload-pre-processing-e2e-extmod-a-4bd51b-6b775cddf9-ld82r\" (UID: \"d141422b-120e-4dc4-bfe0-95ab1391d419\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-4bd51b-6b775cddf9-ld82r"
Oct 31 21:52:33.374422 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:33.374388    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a302e86-54e3-4d9f-81ee-350498a9149e-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-4bd51b-7d76c5559f-r978k\" (UID: \"8a302e86-54e3-4d9f-81ee-350498a9149e\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-4bd51b-7d76c5559f-r978k"
Oct 31 21:52:33.374729 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:33.374710    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbjgv\" (UniqueName: \"kubernetes.io/projected/8a302e86-54e3-4d9f-81ee-350498a9149e-kube-api-access-qbjgv\") pod \"payload-processing-e2e-extmod-a-4bd51b-7d76c5559f-r978k\" (UID: \"8a302e86-54e3-4d9f-81ee-350498a9149e\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-4bd51b-7d76c5559f-r978k"
Oct 31 21:52:33.475857 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:33.475775    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbjgv\" (UniqueName: \"kubernetes.io/projected/8a302e86-54e3-4d9f-81ee-350498a9149e-kube-api-access-qbjgv\") pod \"payload-processing-e2e-extmod-a-4bd51b-7d76c5559f-r978k\" (UID: \"8a302e86-54e3-4d9f-81ee-350498a9149e\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-4bd51b-7d76c5559f-r978k"
Oct 31 21:52:33.475857 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:33.475847    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a302e86-54e3-4d9f-81ee-350498a9149e-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-4bd51b-7d76c5559f-r978k\" (UID: \"8a302e86-54e3-4d9f-81ee-350498a9149e\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-4bd51b-7d76c5559f-r978k"
Oct 31 21:52:33.476643 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:33.476621    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a302e86-54e3-4d9f-81ee-350498a9149e-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-4bd51b-7d76c5559f-r978k\" (UID: \"8a302e86-54e3-4d9f-81ee-350498a9149e\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-4bd51b-7d76c5559f-r978k"
Oct 31 21:52:33.501298 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:33.501273    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbjgv\" (UniqueName: \"kubernetes.io/projected/8a302e86-54e3-4d9f-81ee-350498a9149e-kube-api-access-qbjgv\") pod \"payload-processing-e2e-extmod-a-4bd51b-7d76c5559f-r978k\" (UID: \"8a302e86-54e3-4d9f-81ee-350498a9149e\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-4bd51b-7d76c5559f-r978k"
Oct 31 21:52:33.615458 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:33.615427    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-4bd51b-6b775cddf9-ld82r"
Oct 31 21:52:33.653782 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:33.653751    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:33.655213 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:33.655186    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-4bd51b-7d76c5559f-r978k"
Oct 31 21:52:33.993702 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:33.993675    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-4bd51b-6b775cddf9-ld82r"]
Oct 31 21:52:33.995060 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:52:33.995029    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd141422b_120e_4dc4_bfe0_95ab1391d419.slice/crio-811d7a5891fe7ee14951255435b5541b41f7799ad8aea342bfa396637d3c9f10 WatchSource:0}: Error finding container 811d7a5891fe7ee14951255435b5541b41f7799ad8aea342bfa396637d3c9f10: Status 404 returned error can't find the container with id 811d7a5891fe7ee14951255435b5541b41f7799ad8aea342bfa396637d3c9f10
Oct 31 21:52:34.049169 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:34.049143    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-4bd51b-7d76c5559f-r978k"]
Oct 31 21:52:34.053207 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:52:34.053183    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a302e86_54e3_4d9f_81ee_350498a9149e.slice/crio-5ad0f3a730f189e8e679885df3ed747bd9b2fced8e63453f6ec0755b851a41c2 WatchSource:0}: Error finding container 5ad0f3a730f189e8e679885df3ed747bd9b2fced8e63453f6ec0755b851a41c2: Status 404 returned error can't find the container with id 5ad0f3a730f189e8e679885df3ed747bd9b2fced8e63453f6ec0755b851a41c2
Oct 31 21:52:34.770581 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:34.770537    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-4bd51b-7d76c5559f-r978k" event={"ID":"8a302e86-54e3-4d9f-81ee-350498a9149e","Type":"ContainerStarted","Data":"d0d6a4d80081d692669527d2888b8d6d3d7a32a0b62eed694d1c8e1729c94cd6"}
Oct 31 21:52:34.770779 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:34.770612    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-4bd51b-7d76c5559f-r978k" event={"ID":"8a302e86-54e3-4d9f-81ee-350498a9149e","Type":"ContainerStarted","Data":"5ad0f3a730f189e8e679885df3ed747bd9b2fced8e63453f6ec0755b851a41c2"}
Oct 31 21:52:34.770779 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:34.770642    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-4bd51b-7d76c5559f-r978k"
Oct 31 21:52:34.772170 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:34.772143    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-4bd51b-6b775cddf9-ld82r" event={"ID":"d141422b-120e-4dc4-bfe0-95ab1391d419","Type":"ContainerStarted","Data":"cc401682207c2ee2ce2bcd708a47fb23d318681923efd79f0f962c1d210a69aa"}
Oct 31 21:52:34.772275 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:34.772193    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-4bd51b-6b775cddf9-ld82r" event={"ID":"d141422b-120e-4dc4-bfe0-95ab1391d419","Type":"ContainerStarted","Data":"811d7a5891fe7ee14951255435b5541b41f7799ad8aea342bfa396637d3c9f10"}
Oct 31 21:52:34.772348 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:34.772287    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-4bd51b-6b775cddf9-ld82r"
Oct 31 21:52:34.815345 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:34.815300    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-4bd51b-7d76c5559f-r978k" podStartSLOduration=1.8152866410000001 podStartE2EDuration="1.815286641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:52:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:52:34.81457804 +0000 UTC m=+2194.730887470" watchObservedRunningTime="2026-10-31 21:52:34.815286641 +0000 UTC m=+2194.731596109"
Oct 31 21:52:34.861856 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:34.861804    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-4bd51b-6b775cddf9-ld82r" podStartSLOduration=2.86178813 podStartE2EDuration="2.86178813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:52:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:52:34.858074331 +0000 UTC m=+2194.774383761" watchObservedRunningTime="2026-10-31 21:52:34.86178813 +0000 UTC m=+2194.778097558"
Oct 31 21:52:35.429457 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:35.429421    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:36.808656 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:36.808621    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:52:37.127400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:37.127294    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:37.289195 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:37.289160    4460 patch_prober.go:29] interesting pod/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 21:52:37.289378 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:37.289219    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7" podUID="b06d985f-9a38-4ea8-a120-676d0c3de8aa" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 21:52:38.278889 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.278856    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:52:38.384551 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.384516    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b06d985f-9a38-4ea8-a120-676d0c3de8aa-kube-api-access-8g6pr\" (UniqueName: \"kubernetes.io/projected/b06d985f-9a38-4ea8-a120-676d0c3de8aa-kube-api-access-8g6pr\") pod \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") "
Oct 31 21:52:38.384687 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.384623    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-credential-socket\") pod \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") "
Oct 31 21:52:38.384767 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.384685    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istio-envoy\") pod \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") "
Oct 31 21:52:38.384866 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.384784    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istio-data\") pod \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") "
Oct 31 21:52:38.384866 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.384810    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-workload-certs\") pod \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") "
Oct 31 21:52:38.384866 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.384849    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istiod-ca-cert\") pod \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") "
Oct 31 21:52:38.385174 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.384887    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istio-token\" (UniqueName: \"kubernetes.io/projected/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istio-token\") pod \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") "
Oct 31 21:52:38.385174 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.384978    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-workload-socket\") pod \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") "
Oct 31 21:52:38.385174 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.385023    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istio-podinfo\") pod \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\" (UID: \"b06d985f-9a38-4ea8-a120-676d0c3de8aa\") "
Oct 31 21:52:38.385338 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.385176    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-workload-certs" pod "b06d985f-9a38-4ea8-a120-676d0c3de8aa" (UID: "b06d985f-9a38-4ea8-a120-676d0c3de8aa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:52:38.385876 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.385700    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-workload-socket" pod "b06d985f-9a38-4ea8-a120-676d0c3de8aa" (UID: "b06d985f-9a38-4ea8-a120-676d0c3de8aa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:52:38.386000 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.385787    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.386185 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.386164    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istiod-ca-cert" pod "b06d985f-9a38-4ea8-a120-676d0c3de8aa" (UID: "b06d985f-9a38-4ea8-a120-676d0c3de8aa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:52:38.388225 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.388203    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istio-podinfo" pod "b06d985f-9a38-4ea8-a120-676d0c3de8aa" (UID: "b06d985f-9a38-4ea8-a120-676d0c3de8aa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:52:38.388435 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.388408    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istio-data" pod "b06d985f-9a38-4ea8-a120-676d0c3de8aa" (UID: "b06d985f-9a38-4ea8-a120-676d0c3de8aa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:52:38.388544 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.388524    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-credential-socket" pod "b06d985f-9a38-4ea8-a120-676d0c3de8aa" (UID: "b06d985f-9a38-4ea8-a120-676d0c3de8aa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:52:38.390703 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.390662    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istio-token" pod "b06d985f-9a38-4ea8-a120-676d0c3de8aa" (UID: "b06d985f-9a38-4ea8-a120-676d0c3de8aa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:52:38.392407 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.392088    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:52:38.395656 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.395264    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b06d985f-9a38-4ea8-a120-676d0c3de8aa-kube-api-access-8g6pr" pod "b06d985f-9a38-4ea8-a120-676d0c3de8aa" (UID: "b06d985f-9a38-4ea8-a120-676d0c3de8aa"). InnerVolumeSpecName "kube-api-access-8g6pr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:52:38.397132 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.397110    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istio-envoy" pod "b06d985f-9a38-4ea8-a120-676d0c3de8aa" (UID: "b06d985f-9a38-4ea8-a120-676d0c3de8aa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:52:38.402397 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.402380    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:52:38.407589 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.407572    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:52:38.493017 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.492922    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/96a8801d-ad9e-4268-8dd0-a657e235e83c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96a8801d-ad9e-4268-8dd0-a657e235e83c-istiod-ca-cert\") pod \"96a8801d-ad9e-4268-8dd0-a657e235e83c\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") "
Oct 31 21:52:38.493017 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.492994    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/56d5066a-9abc-4b3d-b905-2250ef807321-istio-token\" (UniqueName: \"kubernetes.io/projected/56d5066a-9abc-4b3d-b905-2250ef807321-istio-token\") pod \"56d5066a-9abc-4b3d-b905-2250ef807321\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") "
Oct 31 21:52:38.493238 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.493030    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/56d5066a-9abc-4b3d-b905-2250ef807321-kube-api-access-nshdr\" (UniqueName: \"kubernetes.io/projected/56d5066a-9abc-4b3d-b905-2250ef807321-kube-api-access-nshdr\") pod \"56d5066a-9abc-4b3d-b905-2250ef807321\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") "
Oct 31 21:52:38.493238 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.493089    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istio-podinfo\") pod \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") "
Oct 31 21:52:38.493238 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.493145    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-workload-certs\") pod \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") "
Oct 31 21:52:38.493238 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.493175    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-workload-socket\") pod \"96a8801d-ad9e-4268-8dd0-a657e235e83c\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") "
Oct 31 21:52:38.493238 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.493235    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-workload-socket\") pod \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") "
Oct 31 21:52:38.493504 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.493265    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-workload-certs\") pod \"96a8801d-ad9e-4268-8dd0-a657e235e83c\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") "
Oct 31 21:52:38.493504 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.493303    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-credential-socket\") pod \"96a8801d-ad9e-4268-8dd0-a657e235e83c\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") "
Oct 31 21:52:38.493504 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.493325    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96a8801d-ad9e-4268-8dd0-a657e235e83c-istiod-ca-cert" pod "96a8801d-ad9e-4268-8dd0-a657e235e83c" (UID: "96a8801d-ad9e-4268-8dd0-a657e235e83c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:52:38.493504 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.493332    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-credential-socket\") pod \"56d5066a-9abc-4b3d-b905-2250ef807321\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") "
Oct 31 21:52:38.493504 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.493382    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/56d5066a-9abc-4b3d-b905-2250ef807321-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56d5066a-9abc-4b3d-b905-2250ef807321-istio-podinfo\") pod \"56d5066a-9abc-4b3d-b905-2250ef807321\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") "
Oct 31 21:52:38.493801 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.493525    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istiod-ca-cert\") pod \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") "
Oct 31 21:52:38.493801 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.493556    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/96a8801d-ad9e-4268-8dd0-a657e235e83c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96a8801d-ad9e-4268-8dd0-a657e235e83c-istio-podinfo\") pod \"96a8801d-ad9e-4268-8dd0-a657e235e83c\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") "
Oct 31 21:52:38.493801 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.493596    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-workload-certs\") pod \"56d5066a-9abc-4b3d-b905-2250ef807321\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") "
Oct 31 21:52:38.493801 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.493628    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-istio-envoy\") pod \"96a8801d-ad9e-4268-8dd0-a657e235e83c\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") "
Oct 31 21:52:38.493801 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.493671    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/96a8801d-ad9e-4268-8dd0-a657e235e83c-kube-api-access-5t4t4\" (UniqueName: \"kubernetes.io/projected/96a8801d-ad9e-4268-8dd0-a657e235e83c-kube-api-access-5t4t4\") pod \"96a8801d-ad9e-4268-8dd0-a657e235e83c\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") "
Oct 31 21:52:38.493801 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.493722    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-istio-envoy\") pod \"56d5066a-9abc-4b3d-b905-2250ef807321\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") "
Oct 31 21:52:38.493801 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.493769    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istio-data\") pod \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") "
Oct 31 21:52:38.493801 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.493797    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istio-token\" (UniqueName: \"kubernetes.io/projected/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istio-token\") pod \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") "
Oct 31 21:52:38.494229 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.493832    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-istio-data\") pod \"56d5066a-9abc-4b3d-b905-2250ef807321\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") "
Oct 31 21:52:38.494229 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.493912    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3996ffdd-3dbe-41f6-ab61-62102a86e76b-kube-api-access-4b5qv\" (UniqueName: \"kubernetes.io/projected/3996ffdd-3dbe-41f6-ab61-62102a86e76b-kube-api-access-4b5qv\") pod \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") "
Oct 31 21:52:38.494229 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.493962    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-credential-socket\") pod \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") "
Oct 31 21:52:38.494229 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.494025    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/96a8801d-ad9e-4268-8dd0-a657e235e83c-istio-token\" (UniqueName: \"kubernetes.io/projected/96a8801d-ad9e-4268-8dd0-a657e235e83c-istio-token\") pod \"96a8801d-ad9e-4268-8dd0-a657e235e83c\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") "
Oct 31 21:52:38.494229 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.494053    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-workload-socket\") pod \"56d5066a-9abc-4b3d-b905-2250ef807321\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") "
Oct 31 21:52:38.495720 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.495690    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/56d5066a-9abc-4b3d-b905-2250ef807321-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56d5066a-9abc-4b3d-b905-2250ef807321-istiod-ca-cert\") pod \"56d5066a-9abc-4b3d-b905-2250ef807321\" (UID: \"56d5066a-9abc-4b3d-b905-2250ef807321\") "
Oct 31 21:52:38.495862 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.495757    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-istio-data\") pod \"96a8801d-ad9e-4268-8dd0-a657e235e83c\" (UID: \"96a8801d-ad9e-4268-8dd0-a657e235e83c\") "
Oct 31 21:52:38.495862 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.495791    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istio-envoy\") pod \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\" (UID: \"3996ffdd-3dbe-41f6-ab61-62102a86e76b\") "
Oct 31 21:52:38.496384 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.493596    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-workload-socket" pod "96a8801d-ad9e-4268-8dd0-a657e235e83c" (UID: "96a8801d-ad9e-4268-8dd0-a657e235e83c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:52:38.496525 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.496507    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istio-data" pod "3996ffdd-3dbe-41f6-ab61-62102a86e76b" (UID: "3996ffdd-3dbe-41f6-ab61-62102a86e76b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:52:38.496615 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.496597    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.496676 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.496623    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.496676 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.496644    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.496676 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.496663    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.496833 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.496677    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.496833 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.496691    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.496833 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.496703    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b06d985f-9a38-4ea8-a120-676d0c3de8aa-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.496833 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.496715    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b06d985f-9a38-4ea8-a120-676d0c3de8aa-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.496833 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.496728    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96a8801d-ad9e-4268-8dd0-a657e235e83c-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.496833 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.496744    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8g6pr\" (UniqueName: \"kubernetes.io/projected/b06d985f-9a38-4ea8-a120-676d0c3de8aa-kube-api-access-8g6pr\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.497163 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.496897    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-credential-socket" pod "96a8801d-ad9e-4268-8dd0-a657e235e83c" (UID: "96a8801d-ad9e-4268-8dd0-a657e235e83c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:52:38.497163 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.497060    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istiod-ca-cert" pod "3996ffdd-3dbe-41f6-ab61-62102a86e76b" (UID: "3996ffdd-3dbe-41f6-ab61-62102a86e76b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:52:38.497859 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.497383    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-workload-certs" pod "3996ffdd-3dbe-41f6-ab61-62102a86e76b" (UID: "3996ffdd-3dbe-41f6-ab61-62102a86e76b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:52:38.497859 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.497575    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-credential-socket" pod "3996ffdd-3dbe-41f6-ab61-62102a86e76b" (UID: "3996ffdd-3dbe-41f6-ab61-62102a86e76b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:52:38.497859 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.497586    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-workload-socket" pod "56d5066a-9abc-4b3d-b905-2250ef807321" (UID: "56d5066a-9abc-4b3d-b905-2250ef807321"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:52:38.498629 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.498598    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istio-envoy" pod "3996ffdd-3dbe-41f6-ab61-62102a86e76b" (UID: "3996ffdd-3dbe-41f6-ab61-62102a86e76b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:52:38.498719 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.498652    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istio-podinfo" pod "3996ffdd-3dbe-41f6-ab61-62102a86e76b" (UID: "3996ffdd-3dbe-41f6-ab61-62102a86e76b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:52:38.498763 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.498734    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56d5066a-9abc-4b3d-b905-2250ef807321-kube-api-access-nshdr" pod "56d5066a-9abc-4b3d-b905-2250ef807321" (UID: "56d5066a-9abc-4b3d-b905-2250ef807321"). InnerVolumeSpecName "kube-api-access-nshdr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:52:38.498880 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.498861    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-istio-envoy" pod "96a8801d-ad9e-4268-8dd0-a657e235e83c" (UID: "96a8801d-ad9e-4268-8dd0-a657e235e83c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:52:38.499014 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.498980    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56d5066a-9abc-4b3d-b905-2250ef807321-istiod-ca-cert" pod "56d5066a-9abc-4b3d-b905-2250ef807321" (UID: "56d5066a-9abc-4b3d-b905-2250ef807321"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:52:38.499695 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.499247    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-istio-data" pod "96a8801d-ad9e-4268-8dd0-a657e235e83c" (UID: "96a8801d-ad9e-4268-8dd0-a657e235e83c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:52:38.499695 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.494905    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-workload-certs" pod "96a8801d-ad9e-4268-8dd0-a657e235e83c" (UID: "96a8801d-ad9e-4268-8dd0-a657e235e83c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:52:38.499695 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.496019    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-workload-socket" pod "3996ffdd-3dbe-41f6-ab61-62102a86e76b" (UID: "3996ffdd-3dbe-41f6-ab61-62102a86e76b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:52:38.499695 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.496198    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-credential-socket" pod "56d5066a-9abc-4b3d-b905-2250ef807321" (UID: "56d5066a-9abc-4b3d-b905-2250ef807321"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:52:38.499695 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.496356    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/56d5066a-9abc-4b3d-b905-2250ef807321-istio-podinfo" pod "56d5066a-9abc-4b3d-b905-2250ef807321" (UID: "56d5066a-9abc-4b3d-b905-2250ef807321"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:52:38.499695 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.499314    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-workload-certs" pod "56d5066a-9abc-4b3d-b905-2250ef807321" (UID: "56d5066a-9abc-4b3d-b905-2250ef807321"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:52:38.499695 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.499392    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/96a8801d-ad9e-4268-8dd0-a657e235e83c-istio-podinfo" pod "96a8801d-ad9e-4268-8dd0-a657e235e83c" (UID: "96a8801d-ad9e-4268-8dd0-a657e235e83c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:52:38.499695 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.499504    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-istio-data" pod "56d5066a-9abc-4b3d-b905-2250ef807321" (UID: "56d5066a-9abc-4b3d-b905-2250ef807321"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:52:38.500199 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.499828    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56d5066a-9abc-4b3d-b905-2250ef807321-istio-token" pod "56d5066a-9abc-4b3d-b905-2250ef807321" (UID: "56d5066a-9abc-4b3d-b905-2250ef807321"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:52:38.500459 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.500436    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96a8801d-ad9e-4268-8dd0-a657e235e83c-kube-api-access-5t4t4" pod "96a8801d-ad9e-4268-8dd0-a657e235e83c" (UID: "96a8801d-ad9e-4268-8dd0-a657e235e83c"). InnerVolumeSpecName "kube-api-access-5t4t4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:52:38.500550 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.500533    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istio-token" pod "3996ffdd-3dbe-41f6-ab61-62102a86e76b" (UID: "3996ffdd-3dbe-41f6-ab61-62102a86e76b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:52:38.501294 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.501276    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3996ffdd-3dbe-41f6-ab61-62102a86e76b-kube-api-access-4b5qv" pod "3996ffdd-3dbe-41f6-ab61-62102a86e76b" (UID: "3996ffdd-3dbe-41f6-ab61-62102a86e76b"). InnerVolumeSpecName "kube-api-access-4b5qv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:52:38.501432 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.501417    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-istio-envoy" pod "56d5066a-9abc-4b3d-b905-2250ef807321" (UID: "56d5066a-9abc-4b3d-b905-2250ef807321"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:52:38.501862 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.501832    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96a8801d-ad9e-4268-8dd0-a657e235e83c-istio-token" pod "96a8801d-ad9e-4268-8dd0-a657e235e83c" (UID: "96a8801d-ad9e-4268-8dd0-a657e235e83c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:52:38.598044 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.598009    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.598044 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.598042    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.598246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.598056    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.598246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.598068    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.598246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.598084    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56d5066a-9abc-4b3d-b905-2250ef807321-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.598246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.598096    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.598246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.598108    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96a8801d-ad9e-4268-8dd0-a657e235e83c-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.598246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.598120    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.598246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.598133    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.598246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.598145    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5t4t4\" (UniqueName: \"kubernetes.io/projected/96a8801d-ad9e-4268-8dd0-a657e235e83c-kube-api-access-5t4t4\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.598246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.598157    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.598246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.598170    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.598246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.598181    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.598246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.598196    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.598246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.598206    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4b5qv\" (UniqueName: \"kubernetes.io/projected/3996ffdd-3dbe-41f6-ab61-62102a86e76b-kube-api-access-4b5qv\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.598246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.598215    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.598246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.598223    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96a8801d-ad9e-4268-8dd0-a657e235e83c-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.598246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.598231    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56d5066a-9abc-4b3d-b905-2250ef807321-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.598246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.598239    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56d5066a-9abc-4b3d-b905-2250ef807321-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.598246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.598247    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96a8801d-ad9e-4268-8dd0-a657e235e83c-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.598246 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.598255    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.598847 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.598262    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56d5066a-9abc-4b3d-b905-2250ef807321-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.598847 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.598274    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nshdr\" (UniqueName: \"kubernetes.io/projected/56d5066a-9abc-4b3d-b905-2250ef807321-kube-api-access-nshdr\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.598847 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.598287    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3996ffdd-3dbe-41f6-ab61-62102a86e76b-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.598847 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.598299    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3996ffdd-3dbe-41f6-ab61-62102a86e76b-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:52:38.799500 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.799461    4460 generic.go:356] "Generic (PLEG): container finished" podID="b06d985f-9a38-4ea8-a120-676d0c3de8aa" containerID="47718e5e2043427be3b2865aab093df5294356bbbb85f14b0aa7551f379ea9f0" exitCode=0
Oct 31 21:52:38.799709 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.799513    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"
Oct 31 21:52:38.799709 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.799542    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7" event={"ID":"b06d985f-9a38-4ea8-a120-676d0c3de8aa","Type":"ContainerDied","Data":"47718e5e2043427be3b2865aab093df5294356bbbb85f14b0aa7551f379ea9f0"}
Oct 31 21:52:38.799709 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.799599    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7" event={"ID":"b06d985f-9a38-4ea8-a120-676d0c3de8aa","Type":"ContainerDied","Data":"7ae70903fc3dfaf53fad680e06fd8769513ea49b1cdcbc21d5c255a66682b41d"}
Oct 31 21:52:38.799709 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.799637    4460 scope.go:122] "RemoveContainer" containerID="47718e5e2043427be3b2865aab093df5294356bbbb85f14b0aa7551f379ea9f0"
Oct 31 21:52:38.801128 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.801102    4460 generic.go:356] "Generic (PLEG): container finished" podID="56d5066a-9abc-4b3d-b905-2250ef807321" containerID="c0b783b082ea28c91ea5400013bbbb4ca732339f5eee490598fd577660a3b4d5" exitCode=0
Oct 31 21:52:38.801224 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.801144    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"
Oct 31 21:52:38.801224 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.801179    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm" event={"ID":"56d5066a-9abc-4b3d-b905-2250ef807321","Type":"ContainerDied","Data":"c0b783b082ea28c91ea5400013bbbb4ca732339f5eee490598fd577660a3b4d5"}
Oct 31 21:52:38.801224 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.801222    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm" event={"ID":"56d5066a-9abc-4b3d-b905-2250ef807321","Type":"ContainerDied","Data":"d60fbf9340d5474808cb55271ee84fe0c1ce9f72a5f8f7984546541ace0f2ad2"}
Oct 31 21:52:38.802789 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.802748    4460 generic.go:356] "Generic (PLEG): container finished" podID="3996ffdd-3dbe-41f6-ab61-62102a86e76b" containerID="860bf94ed996535b0710e5576dc042d5c8eb988d8fcd7ad646f2c39412a662d6" exitCode=0
Oct 31 21:52:38.802789 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.802785    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"
Oct 31 21:52:38.802987 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.802801    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm" event={"ID":"3996ffdd-3dbe-41f6-ab61-62102a86e76b","Type":"ContainerDied","Data":"860bf94ed996535b0710e5576dc042d5c8eb988d8fcd7ad646f2c39412a662d6"}
Oct 31 21:52:38.802987 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.802862    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm" event={"ID":"3996ffdd-3dbe-41f6-ab61-62102a86e76b","Type":"ContainerDied","Data":"fe55643b78f726637105bf3a9f1b640504e049150d24af079c91d7a6f5c28bd8"}
Oct 31 21:52:38.804611 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.804588    4460 generic.go:356] "Generic (PLEG): container finished" podID="96a8801d-ad9e-4268-8dd0-a657e235e83c" containerID="0c63ce52d24a8104077be1e4117001078753b5160412fecf19fc55adde974036" exitCode=0
Oct 31 21:52:38.804704 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.804636    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"
Oct 31 21:52:38.804704 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.804666    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g" event={"ID":"96a8801d-ad9e-4268-8dd0-a657e235e83c","Type":"ContainerDied","Data":"0c63ce52d24a8104077be1e4117001078753b5160412fecf19fc55adde974036"}
Oct 31 21:52:38.804795 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.804707    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g" event={"ID":"96a8801d-ad9e-4268-8dd0-a657e235e83c","Type":"ContainerDied","Data":"68b11d2dbde9c21f6197d562bf6d9139e611428085704a6fca038957298c9a75"}
Oct 31 21:52:38.813088 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.813065    4460 scope.go:122] "RemoveContainer" containerID="47718e5e2043427be3b2865aab093df5294356bbbb85f14b0aa7551f379ea9f0"
Oct 31 21:52:38.813408 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:52:38.813384    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47718e5e2043427be3b2865aab093df5294356bbbb85f14b0aa7551f379ea9f0\": container with ID starting with 47718e5e2043427be3b2865aab093df5294356bbbb85f14b0aa7551f379ea9f0 not found: ID does not exist" containerID="47718e5e2043427be3b2865aab093df5294356bbbb85f14b0aa7551f379ea9f0"
Oct 31 21:52:38.813510 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.813429    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47718e5e2043427be3b2865aab093df5294356bbbb85f14b0aa7551f379ea9f0"} err="failed to get container status \"47718e5e2043427be3b2865aab093df5294356bbbb85f14b0aa7551f379ea9f0\": rpc error: code = NotFound desc = could not find container \"47718e5e2043427be3b2865aab093df5294356bbbb85f14b0aa7551f379ea9f0\": container with ID starting with 47718e5e2043427be3b2865aab093df5294356bbbb85f14b0aa7551f379ea9f0 not found: ID does not exist"
Oct 31 21:52:38.813510 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.813454    4460 scope.go:122] "RemoveContainer" containerID="c0b783b082ea28c91ea5400013bbbb4ca732339f5eee490598fd577660a3b4d5"
Oct 31 21:52:38.833389 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.832226    4460 scope.go:122] "RemoveContainer" containerID="c0b783b082ea28c91ea5400013bbbb4ca732339f5eee490598fd577660a3b4d5"
Oct 31 21:52:38.833389 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:52:38.833242    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0b783b082ea28c91ea5400013bbbb4ca732339f5eee490598fd577660a3b4d5\": container with ID starting with c0b783b082ea28c91ea5400013bbbb4ca732339f5eee490598fd577660a3b4d5 not found: ID does not exist" containerID="c0b783b082ea28c91ea5400013bbbb4ca732339f5eee490598fd577660a3b4d5"
Oct 31 21:52:38.833389 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.833273    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0b783b082ea28c91ea5400013bbbb4ca732339f5eee490598fd577660a3b4d5"} err="failed to get container status \"c0b783b082ea28c91ea5400013bbbb4ca732339f5eee490598fd577660a3b4d5\": rpc error: code = NotFound desc = could not find container \"c0b783b082ea28c91ea5400013bbbb4ca732339f5eee490598fd577660a3b4d5\": container with ID starting with c0b783b082ea28c91ea5400013bbbb4ca732339f5eee490598fd577660a3b4d5 not found: ID does not exist"
Oct 31 21:52:38.833389 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.833294    4460 scope.go:122] "RemoveContainer" containerID="860bf94ed996535b0710e5576dc042d5c8eb988d8fcd7ad646f2c39412a662d6"
Oct 31 21:52:38.843033 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.843014    4460 scope.go:122] "RemoveContainer" containerID="860bf94ed996535b0710e5576dc042d5c8eb988d8fcd7ad646f2c39412a662d6"
Oct 31 21:52:38.843348 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:52:38.843327    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"860bf94ed996535b0710e5576dc042d5c8eb988d8fcd7ad646f2c39412a662d6\": container with ID starting with 860bf94ed996535b0710e5576dc042d5c8eb988d8fcd7ad646f2c39412a662d6 not found: ID does not exist" containerID="860bf94ed996535b0710e5576dc042d5c8eb988d8fcd7ad646f2c39412a662d6"
Oct 31 21:52:38.843406 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.843355    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"860bf94ed996535b0710e5576dc042d5c8eb988d8fcd7ad646f2c39412a662d6"} err="failed to get container status \"860bf94ed996535b0710e5576dc042d5c8eb988d8fcd7ad646f2c39412a662d6\": rpc error: code = NotFound desc = could not find container \"860bf94ed996535b0710e5576dc042d5c8eb988d8fcd7ad646f2c39412a662d6\": container with ID starting with 860bf94ed996535b0710e5576dc042d5c8eb988d8fcd7ad646f2c39412a662d6 not found: ID does not exist"
Oct 31 21:52:38.843406 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.843370    4460 scope.go:122] "RemoveContainer" containerID="0c63ce52d24a8104077be1e4117001078753b5160412fecf19fc55adde974036"
Oct 31 21:52:38.844787 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.844763    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:38.876454 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.876435    4460 scope.go:122] "RemoveContainer" containerID="0c63ce52d24a8104077be1e4117001078753b5160412fecf19fc55adde974036"
Oct 31 21:52:38.876758 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:52:38.876735    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c63ce52d24a8104077be1e4117001078753b5160412fecf19fc55adde974036\": container with ID starting with 0c63ce52d24a8104077be1e4117001078753b5160412fecf19fc55adde974036 not found: ID does not exist" containerID="0c63ce52d24a8104077be1e4117001078753b5160412fecf19fc55adde974036"
Oct 31 21:52:38.876822 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.876768    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c63ce52d24a8104077be1e4117001078753b5160412fecf19fc55adde974036"} err="failed to get container status \"0c63ce52d24a8104077be1e4117001078753b5160412fecf19fc55adde974036\": rpc error: code = NotFound desc = could not find container \"0c63ce52d24a8104077be1e4117001078753b5160412fecf19fc55adde974036\": container with ID starting with 0c63ce52d24a8104077be1e4117001078753b5160412fecf19fc55adde974036 not found: ID does not exist"
Oct 31 21:52:38.925187 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.925145    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"]
Oct 31 21:52:38.933350 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.933317    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-k7cm7"]
Oct 31 21:52:38.966502 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.966460    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"]
Oct 31 21:52:38.973266 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:38.973238    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-hp95g"]
Oct 31 21:52:39.010590 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:39.010553    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"]
Oct 31 21:52:39.015428 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:39.015396    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-qcplm"]
Oct 31 21:52:39.052956 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:39.052866    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"]
Oct 31 21:52:39.059095 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:39.059070    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-4b5989-openshift-default-578774676c-l6flm"]
Oct 31 21:52:40.225845 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:40.225814    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3996ffdd-3dbe-41f6-ab61-62102a86e76b" path="/var/lib/kubelet/pods/3996ffdd-3dbe-41f6-ab61-62102a86e76b/volumes"
Oct 31 21:52:40.226388 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:40.226370    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="56d5066a-9abc-4b3d-b905-2250ef807321" path="/var/lib/kubelet/pods/56d5066a-9abc-4b3d-b905-2250ef807321/volumes"
Oct 31 21:52:40.226834 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:40.226820    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="96a8801d-ad9e-4268-8dd0-a657e235e83c" path="/var/lib/kubelet/pods/96a8801d-ad9e-4268-8dd0-a657e235e83c/volumes"
Oct 31 21:52:40.227585 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:40.227569    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b06d985f-9a38-4ea8-a120-676d0c3de8aa" path="/var/lib/kubelet/pods/b06d985f-9a38-4ea8-a120-676d0c3de8aa/volumes"
Oct 31 21:52:40.477139 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:40.477043    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:42.132916 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:42.132880    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:43.072285 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:43.072204    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:43.952839 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:43.952796    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:44.679395 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:44.679361    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:45.098351 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.098311    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"]
Oct 31 21:52:45.099301 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.099277    4460 cpu_manager.go:414] "containerMap: removing container" podUID="96a8801d-ad9e-4268-8dd0-a657e235e83c" containerName="istio-proxy"
Oct 31 21:52:45.099378 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.099304    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="96a8801d-ad9e-4268-8dd0-a657e235e83c" containerName="istio-proxy"
Oct 31 21:52:45.099413 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.099377    4460 cpu_manager.go:414] "containerMap: removing container" podUID="3996ffdd-3dbe-41f6-ab61-62102a86e76b" containerName="istio-proxy"
Oct 31 21:52:45.099413 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.099386    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3996ffdd-3dbe-41f6-ab61-62102a86e76b" containerName="istio-proxy"
Oct 31 21:52:45.099413 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.099398    4460 cpu_manager.go:414] "containerMap: removing container" podUID="56d5066a-9abc-4b3d-b905-2250ef807321" containerName="istio-proxy"
Oct 31 21:52:45.099413 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.099407    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="56d5066a-9abc-4b3d-b905-2250ef807321" containerName="istio-proxy"
Oct 31 21:52:45.099535 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.099424    4460 cpu_manager.go:414] "containerMap: removing container" podUID="b06d985f-9a38-4ea8-a120-676d0c3de8aa" containerName="istio-proxy"
Oct 31 21:52:45.099535 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.099431    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b06d985f-9a38-4ea8-a120-676d0c3de8aa" containerName="istio-proxy"
Oct 31 21:52:45.099593 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.099559    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="3996ffdd-3dbe-41f6-ab61-62102a86e76b" containerName="istio-proxy"
Oct 31 21:52:45.099593 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.099571    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="56d5066a-9abc-4b3d-b905-2250ef807321" containerName="istio-proxy"
Oct 31 21:52:45.099593 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.099579    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="b06d985f-9a38-4ea8-a120-676d0c3de8aa" containerName="istio-proxy"
Oct 31 21:52:45.099593 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.099586    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="96a8801d-ad9e-4268-8dd0-a657e235e83c" containerName="istio-proxy"
Oct 31 21:52:45.105167 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.105145    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:52:45.121888 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.121854    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-943963-openshift-default-dockercfg-ntxzf\""
Oct 31 21:52:45.140846 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.140815    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"]
Oct 31 21:52:45.166072 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.166037    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:45.198548 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.198514    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istio-podinfo\") pod \"e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") " pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:52:45.198691 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.198635    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istiod-ca-cert\") pod \"e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") " pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:52:45.198741 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.198696    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-workload-socket\") pod \"e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") " pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:52:45.198795 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.198781    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-credential-socket\") pod \"e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") " pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:52:45.198892 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.198871    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istio-data\") pod \"e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") " pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:52:45.198981 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.198918    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istio-token\") pod \"e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") " pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:52:45.199043 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.199027    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-workload-certs\") pod \"e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") " pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:52:45.199100 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.199079    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istio-envoy\") pod \"e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") " pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:52:45.199142 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.199126    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccfnq\" (UniqueName: \"kubernetes.io/projected/b07cb206-7fc8-42a4-9086-dac5ce1d7019-kube-api-access-ccfnq\") pod \"e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") " pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:52:45.305962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.300249    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istiod-ca-cert\") pod \"e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") " pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:52:45.305962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.300307    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-workload-socket\") pod \"e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") " pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:52:45.305962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.300407    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-credential-socket\") pod \"e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") " pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:52:45.305962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.300489    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istio-data\") pod \"e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") " pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:52:45.305962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.300534    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istio-token\") pod \"e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") " pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:52:45.305962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.300580    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-workload-certs\") pod \"e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") " pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:52:45.305962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.300623    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istio-envoy\") pod \"e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") " pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:52:45.305962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.300662    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccfnq\" (UniqueName: \"kubernetes.io/projected/b07cb206-7fc8-42a4-9086-dac5ce1d7019-kube-api-access-ccfnq\") pod \"e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") " pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:52:45.305962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.300753    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istio-podinfo\") pod \"e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") " pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:52:45.305962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.301640    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istio-data\") pod \"e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") " pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:52:45.316959 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.314213    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-credential-socket\") pod \"e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") " pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:52:45.316959 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.314507    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-workload-socket\") pod \"e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") " pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:52:45.316959 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.314783    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-workload-certs\") pod \"e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") " pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:52:45.325149 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.323383    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istiod-ca-cert\") pod \"e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") " pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:52:45.336768 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.336706    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istio-podinfo\") pod \"e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") " pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:52:45.345963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.342744    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istio-envoy\") pod \"e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") " pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:52:45.363546 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.363470    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istio-token\") pod \"e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") " pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:52:45.365077 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.365043    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:45.367000 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.366198    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccfnq\" (UniqueName: \"kubernetes.io/projected/b07cb206-7fc8-42a4-9086-dac5ce1d7019-kube-api-access-ccfnq\") pod \"e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") " pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:52:45.424963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.424331    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:52:45.769977 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.769924    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"]
Oct 31 21:52:45.770314 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:52:45.770289    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb07cb206_7fc8_42a4_9086_dac5ce1d7019.slice/crio-d245ddbabfbd55465cd0cf942e235c83b5a41b8075f2f89047ac729d493740d6 WatchSource:0}: Error finding container d245ddbabfbd55465cd0cf942e235c83b5a41b8075f2f89047ac729d493740d6: Status 404 returned error can't find the container with id d245ddbabfbd55465cd0cf942e235c83b5a41b8075f2f89047ac729d493740d6
Oct 31 21:52:45.773239 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.773205    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:52:45.773338 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.773290    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:52:45.773395 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.773364    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:52:45.781373 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.781353    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-4bd51b-7d76c5559f-r978k"
Oct 31 21:52:45.781469 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.781424    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-4bd51b-6b775cddf9-ld82r"
Oct 31 21:52:45.807649 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.807632    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:45.882913 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.882876    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf" event={"ID":"b07cb206-7fc8-42a4-9086-dac5ce1d7019","Type":"ContainerStarted","Data":"03dcb7cdafbd70b4db5bada1f9d3ca6ab727fec4ccd5325598f97683283b6967"}
Oct 31 21:52:45.883097 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.883020    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf" event={"ID":"b07cb206-7fc8-42a4-9086-dac5ce1d7019","Type":"ContainerStarted","Data":"d245ddbabfbd55465cd0cf942e235c83b5a41b8075f2f89047ac729d493740d6"}
Oct 31 21:52:45.939590 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:45.939523    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf" podStartSLOduration=0.939504644 podStartE2EDuration="939.504644ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:52:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:52:45.934354399 +0000 UTC m=+2205.850663827" watchObservedRunningTime="2026-10-31 21:52:45.939504644 +0000 UTC m=+2205.855814076"
Oct 31 21:52:46.426031 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:46.426000    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:52:46.431630 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:46.431606    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:52:46.889169 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:46.889133    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:52:46.890028 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:46.890008    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:52:47.528610 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:47.528570    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:48.045839 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:48.045802    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:48.153391 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:48.153355    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:48.304442 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:48.304360    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:48.413472 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:48.413436    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:49.437264 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:49.437216    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:50.046796 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:50.046757    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:50.156294 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:50.156258    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:50.236962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:50.236914    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-943963-688574d587-7wwhp"]
Oct 31 21:52:50.241274 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:50.241251    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-943963-688574d587-7wwhp"
Oct 31 21:52:50.260157 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:50.260135    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-943963-688574d587-7wwhp"]
Oct 31 21:52:50.270032 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:50.270010    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-943963\""
Oct 31 21:52:50.270133 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:50.270044    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-943963-dockercfg-htw6s\""
Oct 31 21:52:50.279096 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:50.279065    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfn59\" (UniqueName: \"kubernetes.io/projected/ebe9a6f2-adf2-4256-bbea-963048f4a0ad-kube-api-access-wfn59\") pod \"payload-pre-processing-e2e-extmod-a-943963-688574d587-7wwhp\" (UID: \"ebe9a6f2-adf2-4256-bbea-963048f4a0ad\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-943963-688574d587-7wwhp"
Oct 31 21:52:50.279370 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:50.279348    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebe9a6f2-adf2-4256-bbea-963048f4a0ad-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-943963-688574d587-7wwhp\" (UID: \"ebe9a6f2-adf2-4256-bbea-963048f4a0ad\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-943963-688574d587-7wwhp"
Oct 31 21:52:50.435572 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:50.385985    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebe9a6f2-adf2-4256-bbea-963048f4a0ad-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-943963-688574d587-7wwhp\" (UID: \"ebe9a6f2-adf2-4256-bbea-963048f4a0ad\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-943963-688574d587-7wwhp"
Oct 31 21:52:50.525094 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:50.491130    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebe9a6f2-adf2-4256-bbea-963048f4a0ad-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-943963-688574d587-7wwhp\" (UID: \"ebe9a6f2-adf2-4256-bbea-963048f4a0ad\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-943963-688574d587-7wwhp"
Oct 31 21:52:50.525547 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:50.525154    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfn59\" (UniqueName: \"kubernetes.io/projected/ebe9a6f2-adf2-4256-bbea-963048f4a0ad-kube-api-access-wfn59\") pod \"payload-pre-processing-e2e-extmod-a-943963-688574d587-7wwhp\" (UID: \"ebe9a6f2-adf2-4256-bbea-963048f4a0ad\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-943963-688574d587-7wwhp"
Oct 31 21:52:50.533976 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:50.532001    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-943963-7568c9dd8c-99fk7"]
Oct 31 21:52:50.541913 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:50.541889    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-943963-7568c9dd8c-99fk7"
Oct 31 21:52:50.552760 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:50.552740    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfn59\" (UniqueName: \"kubernetes.io/projected/ebe9a6f2-adf2-4256-bbea-963048f4a0ad-kube-api-access-wfn59\") pod \"payload-pre-processing-e2e-extmod-a-943963-688574d587-7wwhp\" (UID: \"ebe9a6f2-adf2-4256-bbea-963048f4a0ad\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-943963-688574d587-7wwhp"
Oct 31 21:52:50.555953 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:50.555920    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-943963-688574d587-7wwhp"
Oct 31 21:52:50.557874 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:50.557846    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-943963-7568c9dd8c-99fk7"]
Oct 31 21:52:50.729064 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:50.728971    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/406f2702-7e47-4467-9f9c-a41f8f9ea53d-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-943963-7568c9dd8c-99fk7\" (UID: \"406f2702-7e47-4467-9f9c-a41f8f9ea53d\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-943963-7568c9dd8c-99fk7"
Oct 31 21:52:50.729064 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:50.729016    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pv9d\" (UniqueName: \"kubernetes.io/projected/406f2702-7e47-4467-9f9c-a41f8f9ea53d-kube-api-access-8pv9d\") pod \"payload-processing-e2e-extmod-a-943963-7568c9dd8c-99fk7\" (UID: \"406f2702-7e47-4467-9f9c-a41f8f9ea53d\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-943963-7568c9dd8c-99fk7"
Oct 31 21:52:50.830412 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:50.830379    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/406f2702-7e47-4467-9f9c-a41f8f9ea53d-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-943963-7568c9dd8c-99fk7\" (UID: \"406f2702-7e47-4467-9f9c-a41f8f9ea53d\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-943963-7568c9dd8c-99fk7"
Oct 31 21:52:50.830593 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:50.830419    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pv9d\" (UniqueName: \"kubernetes.io/projected/406f2702-7e47-4467-9f9c-a41f8f9ea53d-kube-api-access-8pv9d\") pod \"payload-processing-e2e-extmod-a-943963-7568c9dd8c-99fk7\" (UID: \"406f2702-7e47-4467-9f9c-a41f8f9ea53d\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-943963-7568c9dd8c-99fk7"
Oct 31 21:52:50.831040 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:50.831020    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/406f2702-7e47-4467-9f9c-a41f8f9ea53d-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-943963-7568c9dd8c-99fk7\" (UID: \"406f2702-7e47-4467-9f9c-a41f8f9ea53d\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-943963-7568c9dd8c-99fk7"
Oct 31 21:52:50.855905 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:50.855877    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pv9d\" (UniqueName: \"kubernetes.io/projected/406f2702-7e47-4467-9f9c-a41f8f9ea53d-kube-api-access-8pv9d\") pod \"payload-processing-e2e-extmod-a-943963-7568c9dd8c-99fk7\" (UID: \"406f2702-7e47-4467-9f9c-a41f8f9ea53d\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-943963-7568c9dd8c-99fk7"
Oct 31 21:52:50.867511 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:50.867488    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-943963-688574d587-7wwhp"]
Oct 31 21:52:50.871899 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:52:50.871876    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebe9a6f2_adf2_4256_bbea_963048f4a0ad.slice/crio-55cfc8730b6540db5c214ebdf058190802730029d119aba8becadd86a02519b9 WatchSource:0}: Error finding container 55cfc8730b6540db5c214ebdf058190802730029d119aba8becadd86a02519b9: Status 404 returned error can't find the container with id 55cfc8730b6540db5c214ebdf058190802730029d119aba8becadd86a02519b9
Oct 31 21:52:50.873225 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:50.873205    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-943963-7568c9dd8c-99fk7"
Oct 31 21:52:50.918016 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:50.917980    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-943963-688574d587-7wwhp" event={"ID":"ebe9a6f2-adf2-4256-bbea-963048f4a0ad","Type":"ContainerStarted","Data":"55cfc8730b6540db5c214ebdf058190802730029d119aba8becadd86a02519b9"}
Oct 31 21:52:51.184767 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:51.184734    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-943963-7568c9dd8c-99fk7"]
Oct 31 21:52:51.184985 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:52:51.184959    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod406f2702_7e47_4467_9f9c_a41f8f9ea53d.slice/crio-0e941c269989753a6645778d57718432c9881dd7c29d32a5e4ea96c69d527d77 WatchSource:0}: Error finding container 0e941c269989753a6645778d57718432c9881dd7c29d32a5e4ea96c69d527d77: Status 404 returned error can't find the container with id 0e941c269989753a6645778d57718432c9881dd7c29d32a5e4ea96c69d527d77
Oct 31 21:52:51.217504 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:51.217478    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:51.452696 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:52:51.452603    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:52:51.926318 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:51.926281    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-943963-688574d587-7wwhp" event={"ID":"ebe9a6f2-adf2-4256-bbea-963048f4a0ad","Type":"ContainerStarted","Data":"288a53ef1d3ce07122f7a7967d0112323073357200626ef55b48d1031ff0aaf5"}
Oct 31 21:52:51.926773 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:51.926375    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-943963-688574d587-7wwhp"
Oct 31 21:52:51.927997 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:51.927975    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-943963-7568c9dd8c-99fk7" event={"ID":"406f2702-7e47-4467-9f9c-a41f8f9ea53d","Type":"ContainerStarted","Data":"6ca7c94dc2bba91d807925df328df4cccc3fa917dda36a58ed9e9db458bc7637"}
Oct 31 21:52:51.928128 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:51.928008    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-943963-7568c9dd8c-99fk7" event={"ID":"406f2702-7e47-4467-9f9c-a41f8f9ea53d","Type":"ContainerStarted","Data":"0e941c269989753a6645778d57718432c9881dd7c29d32a5e4ea96c69d527d77"}
Oct 31 21:52:51.928128 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:51.928096    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-943963-7568c9dd8c-99fk7"
Oct 31 21:52:51.972444 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:51.972397    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-943963-688574d587-7wwhp" podStartSLOduration=1.972384725 podStartE2EDuration="1.972384725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:52:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:52:51.969883433 +0000 UTC m=+2211.886192862" watchObservedRunningTime="2026-10-31 21:52:51.972384725 +0000 UTC m=+2211.888694178"
Oct 31 21:52:52.016410 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:52.016365    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-943963-7568c9dd8c-99fk7" podStartSLOduration=2.016354076 podStartE2EDuration="2.016354076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:52:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:52:52.015343997 +0000 UTC m=+2211.931653425" watchObservedRunningTime="2026-10-31 21:52:52.016354076 +0000 UTC m=+2211.932663504"
Oct 31 21:52:52.919354 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:52.919314    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:54.575953 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:54.575902    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:54.702305 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:54.702251    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:56.322770 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:56.322733    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:58.008166 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:58.008136    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:52:59.742057 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:52:59.742012    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:01.526361 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:01.526325    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:02.937897 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:02.937869    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-943963-688574d587-7wwhp"
Oct 31 21:53:02.938283 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:02.937957    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-943963-7568c9dd8c-99fk7"
Oct 31 21:53:03.228878 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:03.228794    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:04.934524 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:04.934483    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:06.566642 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:06.566611    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:08.216379 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:08.216341    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:09.919235 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:09.919199    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:11.639137 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:11.639099    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:12.064253 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:12.064225    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:13.435244 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:13.435200    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx"]
Oct 31 21:53:13.442513 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:13.442480    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx"
Oct 31 21:53:13.465028 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:13.464999    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx"]
Oct 31 21:53:13.466664 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:13.466645    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-943963-serving-cert\""
Oct 31 21:53:13.480308 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:13.480274    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8c618bb-9eca-482e-8d17-f0f7a1ffd12c-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx\" (UID: \"a8c618bb-9eca-482e-8d17-f0f7a1ffd12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx"
Oct 31 21:53:13.480430 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:13.480341    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8c618bb-9eca-482e-8d17-f0f7a1ffd12c-maas-api-tls\") pod \"maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx\" (UID: \"a8c618bb-9eca-482e-8d17-f0f7a1ffd12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx"
Oct 31 21:53:13.480523 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:13.480508    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlh8r\" (UniqueName: \"kubernetes.io/projected/a8c618bb-9eca-482e-8d17-f0f7a1ffd12c-kube-api-access-xlh8r\") pod \"maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx\" (UID: \"a8c618bb-9eca-482e-8d17-f0f7a1ffd12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx"
Oct 31 21:53:13.480569 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:13.480544    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8c618bb-9eca-482e-8d17-f0f7a1ffd12c-metrics-certs\") pod \"maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx\" (UID: \"a8c618bb-9eca-482e-8d17-f0f7a1ffd12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx"
Oct 31 21:53:13.511432 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:13.511399    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:13.593966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:13.591845    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8c618bb-9eca-482e-8d17-f0f7a1ffd12c-metrics-certs\") pod \"maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx\" (UID: \"a8c618bb-9eca-482e-8d17-f0f7a1ffd12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx"
Oct 31 21:53:13.593966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:13.591964    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8c618bb-9eca-482e-8d17-f0f7a1ffd12c-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx\" (UID: \"a8c618bb-9eca-482e-8d17-f0f7a1ffd12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx"
Oct 31 21:53:13.593966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:13.592045    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8c618bb-9eca-482e-8d17-f0f7a1ffd12c-maas-api-tls\") pod \"maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx\" (UID: \"a8c618bb-9eca-482e-8d17-f0f7a1ffd12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx"
Oct 31 21:53:13.593966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:13.592325    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlh8r\" (UniqueName: \"kubernetes.io/projected/a8c618bb-9eca-482e-8d17-f0f7a1ffd12c-kube-api-access-xlh8r\") pod \"maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx\" (UID: \"a8c618bb-9eca-482e-8d17-f0f7a1ffd12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx"
Oct 31 21:53:13.621962 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:53:13.610669    4460 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-a-943963-serving-cert: secret "maas-api-e2e-extmod-a-943963-serving-cert" not found
Oct 31 21:53:13.621962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:13.610702    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8c618bb-9eca-482e-8d17-f0f7a1ffd12c-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx\" (UID: \"a8c618bb-9eca-482e-8d17-f0f7a1ffd12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx"
Oct 31 21:53:13.621962 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:53:13.610779    4460 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8c618bb-9eca-482e-8d17-f0f7a1ffd12c-maas-api-tls podName:a8c618bb-9eca-482e-8d17-f0f7a1ffd12c nodeName:}" failed. No retries permitted until 2026-10-31 21:53:14.110757175 +0000 UTC m=+2234.027066598 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a8c618bb-9eca-482e-8d17-f0f7a1ffd12c-maas-api-tls") pod "maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx" (UID: "a8c618bb-9eca-482e-8d17-f0f7a1ffd12c") : secret "maas-api-e2e-extmod-a-943963-serving-cert" not found
Oct 31 21:53:13.621962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:13.616512    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8c618bb-9eca-482e-8d17-f0f7a1ffd12c-metrics-certs\") pod \"maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx\" (UID: \"a8c618bb-9eca-482e-8d17-f0f7a1ffd12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx"
Oct 31 21:53:13.655481 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:13.653899    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlh8r\" (UniqueName: \"kubernetes.io/projected/a8c618bb-9eca-482e-8d17-f0f7a1ffd12c-kube-api-access-xlh8r\") pod \"maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx\" (UID: \"a8c618bb-9eca-482e-8d17-f0f7a1ffd12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx"
Oct 31 21:53:13.664374 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:13.664336    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:13.786702 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:13.786670    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:14.199279 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:14.199180    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8c618bb-9eca-482e-8d17-f0f7a1ffd12c-maas-api-tls\") pod \"maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx\" (UID: \"a8c618bb-9eca-482e-8d17-f0f7a1ffd12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx"
Oct 31 21:53:14.202057 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:14.202033    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8c618bb-9eca-482e-8d17-f0f7a1ffd12c-maas-api-tls\") pod \"maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx\" (UID: \"a8c618bb-9eca-482e-8d17-f0f7a1ffd12c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx"
Oct 31 21:53:14.364137 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:14.364103    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx"
Oct 31 21:53:14.676125 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:14.676091    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx"]
Oct 31 21:53:14.677765 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:53:14.677739    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8c618bb_9eca_482e_8d17_f0f7a1ffd12c.slice/crio-e59572b0b968801ec4546fbf858134b5c19dc7ba85e28f58757e0e90536a1bbd WatchSource:0}: Error finding container e59572b0b968801ec4546fbf858134b5c19dc7ba85e28f58757e0e90536a1bbd: Status 404 returned error can't find the container with id e59572b0b968801ec4546fbf858134b5c19dc7ba85e28f58757e0e90536a1bbd
Oct 31 21:53:14.679800 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:14.679779    4460 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 21:53:15.101913 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:15.101885    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx" event={"ID":"a8c618bb-9eca-482e-8d17-f0f7a1ffd12c","Type":"ContainerStarted","Data":"caf95d698a24d76171c54011ee3d8fe7c403fe3e41b1a4d79ab3f2e8815d2bb1"}
Oct 31 21:53:15.102057 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:15.101935    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx" event={"ID":"a8c618bb-9eca-482e-8d17-f0f7a1ffd12c","Type":"ContainerStarted","Data":"e59572b0b968801ec4546fbf858134b5c19dc7ba85e28f58757e0e90536a1bbd"}
Oct 31 21:53:15.102057 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:15.101995    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx"
Oct 31 21:53:15.155626 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:15.155555    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx" podStartSLOduration=1.812142251 podStartE2EDuration="2.155537569s" totalImagesPullingTime="343.395318ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:53:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:53:15.153845509 +0000 UTC m=+2235.070154965" watchObservedRunningTime="2026-10-31 21:53:15.155537569 +0000 UTC m=+2235.071847001"
Oct 31 21:53:15.317507 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:15.317421    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:17.066831 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:17.066790    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:18.065963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:18.065029    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:53:18.900973 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:18.900912    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:20.662625 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:20.662591    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:21.114615 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:21.114584    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx"
Oct 31 21:53:21.481459 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:53:21.481364    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:53:22.366169 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:22.366130    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:24.102876 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:24.102840    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:25.907342 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:25.907302    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:26.941969 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:26.941894    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:27.626624 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:27.626568    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:28.197458 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:28.197419    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:28.664105 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:28.664040    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:28.797149 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:28.797109    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:29.324172 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:29.324136    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:30.245524 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:30.245486    4460 generic.go:356] "Generic (PLEG): container finished" podID="20a6bd1d-39a8-4b3e-898b-bfdd13a155b3" containerID="76cf8b03daa7ffbdbfec18e344a67a0b3659024fdafdcd4780bc651790b61d1d" exitCode=0
Oct 31 21:53:30.245708 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:30.245559    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8" event={"ID":"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3","Type":"ContainerDied","Data":"76cf8b03daa7ffbdbfec18e344a67a0b3659024fdafdcd4780bc651790b61d1d"}
Oct 31 21:53:30.246104 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:30.246087    4460 scope.go:122] "RemoveContainer" containerID="76cf8b03daa7ffbdbfec18e344a67a0b3659024fdafdcd4780bc651790b61d1d"
Oct 31 21:53:30.247775 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:30.247748    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:53:30.247842 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:30.247820    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:53:30.247878 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:30.247856    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:53:30.717830 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:30.717786    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:53:31.016031 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:31.015996    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:31.277174 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:31.277088    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8" event={"ID":"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3","Type":"ContainerStarted","Data":"860533e7e4c6596608c19aedf5a150b06b22f7804b651b043fe7f5eb8741b837"}
Oct 31 21:53:31.766235 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:31.766209    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:53:31.767320 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:31.767298    4460 patch_prober.go:29] interesting pod/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.157:15021/healthz/ready\": dial tcp 10.133.0.157:15021: connect: connection refused" start-of-body=
Oct 31 21:53:31.767432 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:31.767338    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8" podUID="20a6bd1d-39a8-4b3e-898b-bfdd13a155b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.157:15021/healthz/ready\": dial tcp 10.133.0.157:15021: connect: connection refused"
Oct 31 21:53:32.440624 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:32.440555    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:32.554446 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:32.554409    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:32.765859 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:32.765821    4460 patch_prober.go:29] interesting pod/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.157:15021/healthz/ready\": dial tcp 10.133.0.157:15021: connect: connection refused" start-of-body=
Oct 31 21:53:32.766073 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:32.765876    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8" podUID="20a6bd1d-39a8-4b3e-898b-bfdd13a155b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.157:15021/healthz/ready\": dial tcp 10.133.0.157:15021: connect: connection refused"
Oct 31 21:53:32.811070 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:32.811034    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:33.766375 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:33.766342    4460 patch_prober.go:29] interesting pod/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.157:15021/healthz/ready\": dial tcp 10.133.0.157:15021: connect: connection refused" start-of-body=
Oct 31 21:53:33.766570 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:33.766398    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8" podUID="20a6bd1d-39a8-4b3e-898b-bfdd13a155b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.157:15021/healthz/ready\": dial tcp 10.133.0.157:15021: connect: connection refused"
Oct 31 21:53:34.204431 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:34.204344    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:34.364034 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:34.363994    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:34.496144 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:34.496052    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-c34cbd-7dc696c947-v5wpt"]
Oct 31 21:53:34.502292 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:34.502136    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c34cbd-7dc696c947-v5wpt"
Oct 31 21:53:34.535964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:34.525523    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-c34cbd\""
Oct 31 21:53:34.545022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:34.544988    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-c34cbd-559d4ccccf-bxss4"]
Oct 31 21:53:34.563197 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:34.563162    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-c34cbd-7dc696c947-v5wpt"]
Oct 31 21:53:34.563534 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:34.563515    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-c34cbd-559d4ccccf-bxss4"
Oct 31 21:53:34.571964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:34.570275    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-c34cbd-dockercfg-dsqp6\""
Oct 31 21:53:34.665506 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:34.663378    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-c34cbd-559d4ccccf-bxss4"]
Oct 31 21:53:34.703652 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:34.702547    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:34.705264 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:34.705230    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4vqz\" (UniqueName: \"kubernetes.io/projected/e2e149ab-f40b-4d96-91df-b230dfae9e90-kube-api-access-x4vqz\") pod \"payload-processing-e2e-extmod-b-c34cbd-559d4ccccf-bxss4\" (UID: \"e2e149ab-f40b-4d96-91df-b230dfae9e90\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-c34cbd-559d4ccccf-bxss4"
Oct 31 21:53:34.705466 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:34.705445    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a22daeb7-8d30-4dcc-98be-e8f404ab1c42-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-c34cbd-7dc696c947-v5wpt\" (UID: \"a22daeb7-8d30-4dcc-98be-e8f404ab1c42\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c34cbd-7dc696c947-v5wpt"
Oct 31 21:53:34.705570 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:34.705551    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2e149ab-f40b-4d96-91df-b230dfae9e90-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-c34cbd-559d4ccccf-bxss4\" (UID: \"e2e149ab-f40b-4d96-91df-b230dfae9e90\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-c34cbd-559d4ccccf-bxss4"
Oct 31 21:53:34.705641 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:34.705595    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88k8b\" (UniqueName: \"kubernetes.io/projected/a22daeb7-8d30-4dcc-98be-e8f404ab1c42-kube-api-access-88k8b\") pod \"payload-pre-processing-e2e-extmod-b-c34cbd-7dc696c947-v5wpt\" (UID: \"a22daeb7-8d30-4dcc-98be-e8f404ab1c42\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c34cbd-7dc696c947-v5wpt"
Oct 31 21:53:34.871981 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:34.871916    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a22daeb7-8d30-4dcc-98be-e8f404ab1c42-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-c34cbd-7dc696c947-v5wpt\" (UID: \"a22daeb7-8d30-4dcc-98be-e8f404ab1c42\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c34cbd-7dc696c947-v5wpt"
Oct 31 21:53:34.872362 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:34.872336    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2e149ab-f40b-4d96-91df-b230dfae9e90-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-c34cbd-559d4ccccf-bxss4\" (UID: \"e2e149ab-f40b-4d96-91df-b230dfae9e90\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-c34cbd-559d4ccccf-bxss4"
Oct 31 21:53:34.872529 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:34.872513    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88k8b\" (UniqueName: \"kubernetes.io/projected/a22daeb7-8d30-4dcc-98be-e8f404ab1c42-kube-api-access-88k8b\") pod \"payload-pre-processing-e2e-extmod-b-c34cbd-7dc696c947-v5wpt\" (UID: \"a22daeb7-8d30-4dcc-98be-e8f404ab1c42\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c34cbd-7dc696c947-v5wpt"
Oct 31 21:53:34.872764 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:34.872745    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4vqz\" (UniqueName: \"kubernetes.io/projected/e2e149ab-f40b-4d96-91df-b230dfae9e90-kube-api-access-x4vqz\") pod \"payload-processing-e2e-extmod-b-c34cbd-559d4ccccf-bxss4\" (UID: \"e2e149ab-f40b-4d96-91df-b230dfae9e90\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-c34cbd-559d4ccccf-bxss4"
Oct 31 21:53:34.873806 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:34.873781    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2e149ab-f40b-4d96-91df-b230dfae9e90-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-c34cbd-559d4ccccf-bxss4\" (UID: \"e2e149ab-f40b-4d96-91df-b230dfae9e90\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-c34cbd-559d4ccccf-bxss4"
Oct 31 21:53:34.875213 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:34.874424    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a22daeb7-8d30-4dcc-98be-e8f404ab1c42-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-c34cbd-7dc696c947-v5wpt\" (UID: \"a22daeb7-8d30-4dcc-98be-e8f404ab1c42\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c34cbd-7dc696c947-v5wpt"
Oct 31 21:53:34.875631 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:34.875607    4460 patch_prober.go:29] interesting pod/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.157:15021/healthz/ready\": dial tcp 10.133.0.157:15021: connect: connection refused" start-of-body=
Oct 31 21:53:34.875807 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:34.875780    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8" podUID="20a6bd1d-39a8-4b3e-898b-bfdd13a155b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.157:15021/healthz/ready\": dial tcp 10.133.0.157:15021: connect: connection refused"
Oct 31 21:53:34.898507 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:34.898467    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-88k8b\" (UniqueName: \"kubernetes.io/projected/a22daeb7-8d30-4dcc-98be-e8f404ab1c42-kube-api-access-88k8b\") pod \"payload-pre-processing-e2e-extmod-b-c34cbd-7dc696c947-v5wpt\" (UID: \"a22daeb7-8d30-4dcc-98be-e8f404ab1c42\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c34cbd-7dc696c947-v5wpt"
Oct 31 21:53:34.898645 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:34.898592    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4vqz\" (UniqueName: \"kubernetes.io/projected/e2e149ab-f40b-4d96-91df-b230dfae9e90-kube-api-access-x4vqz\") pod \"payload-processing-e2e-extmod-b-c34cbd-559d4ccccf-bxss4\" (UID: \"e2e149ab-f40b-4d96-91df-b230dfae9e90\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-c34cbd-559d4ccccf-bxss4"
Oct 31 21:53:34.948774 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:34.948731    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c34cbd-7dc696c947-v5wpt"
Oct 31 21:53:35.005333 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:35.005290    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-c34cbd-559d4ccccf-bxss4"
Oct 31 21:53:35.265442 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:53:35.265401    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda22daeb7_8d30_4dcc_98be_e8f404ab1c42.slice/crio-dfcd585ac12421b071bc91c1298e549231a910ee2bddeb07b0c27e54eec80a57 WatchSource:0}: Error finding container dfcd585ac12421b071bc91c1298e549231a910ee2bddeb07b0c27e54eec80a57: Status 404 returned error can't find the container with id dfcd585ac12421b071bc91c1298e549231a910ee2bddeb07b0c27e54eec80a57
Oct 31 21:53:35.266427 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:35.266393    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-c34cbd-7dc696c947-v5wpt"]
Oct 31 21:53:35.307049 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:35.306933    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c34cbd-7dc696c947-v5wpt" event={"ID":"a22daeb7-8d30-4dcc-98be-e8f404ab1c42","Type":"ContainerStarted","Data":"dfcd585ac12421b071bc91c1298e549231a910ee2bddeb07b0c27e54eec80a57"}
Oct 31 21:53:35.409669 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:35.409636    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-c34cbd-559d4ccccf-bxss4"]
Oct 31 21:53:35.410166 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:53:35.410140    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2e149ab_f40b_4d96_91df_b230dfae9e90.slice/crio-7f2f6b84ef328713e88e6dc5e2845dbddf4afda629ab60949401f2a140b56b3d WatchSource:0}: Error finding container 7f2f6b84ef328713e88e6dc5e2845dbddf4afda629ab60949401f2a140b56b3d: Status 404 returned error can't find the container with id 7f2f6b84ef328713e88e6dc5e2845dbddf4afda629ab60949401f2a140b56b3d
Oct 31 21:53:35.765906 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:35.765877    4460 patch_prober.go:29] interesting pod/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.157:15021/healthz/ready\": dial tcp 10.133.0.157:15021: connect: connection refused" start-of-body=
Oct 31 21:53:35.766118 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:35.765935    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8" podUID="20a6bd1d-39a8-4b3e-898b-bfdd13a155b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.157:15021/healthz/ready\": dial tcp 10.133.0.157:15021: connect: connection refused"
Oct 31 21:53:36.316196 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:36.316155    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c34cbd-7dc696c947-v5wpt" event={"ID":"a22daeb7-8d30-4dcc-98be-e8f404ab1c42","Type":"ContainerStarted","Data":"821788ddfbf583e3a0219e06a757dda353f9ccd1dd26540ccc71c7a9f3be5a5b"}
Oct 31 21:53:36.316712 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:36.316219    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c34cbd-7dc696c947-v5wpt"
Oct 31 21:53:36.317988 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:36.317930    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-c34cbd-559d4ccccf-bxss4" event={"ID":"e2e149ab-f40b-4d96-91df-b230dfae9e90","Type":"ContainerStarted","Data":"8f7e815b1a572b6f9ed988032d688cd0868ea44374f15808e8082a7d3d093825"}
Oct 31 21:53:36.318121 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:36.318015    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-c34cbd-559d4ccccf-bxss4" event={"ID":"e2e149ab-f40b-4d96-91df-b230dfae9e90","Type":"ContainerStarted","Data":"7f2f6b84ef328713e88e6dc5e2845dbddf4afda629ab60949401f2a140b56b3d"}
Oct 31 21:53:36.318121 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:36.318047    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-c34cbd-559d4ccccf-bxss4"
Oct 31 21:53:36.368214 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:36.368160    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c34cbd-7dc696c947-v5wpt" podStartSLOduration=2.368145714 podStartE2EDuration="2.368145714s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:53:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:53:36.366061874 +0000 UTC m=+2256.282371305" watchObservedRunningTime="2026-10-31 21:53:36.368145714 +0000 UTC m=+2256.284455143"
Oct 31 21:53:36.411493 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:36.411437    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-b-c34cbd-559d4ccccf-bxss4" podStartSLOduration=2.411417507 podStartE2EDuration="2.411417507s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:53:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:53:36.409244293 +0000 UTC m=+2256.325553723" watchObservedRunningTime="2026-10-31 21:53:36.411417507 +0000 UTC m=+2256.327726937"
Oct 31 21:53:36.445586 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:36.445548    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:36.768812 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:36.768354    4460 patch_prober.go:29] interesting pod/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.157:15021/healthz/ready\": dial tcp 10.133.0.157:15021: connect: connection refused" start-of-body=
Oct 31 21:53:36.768812 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:36.768418    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8" podUID="20a6bd1d-39a8-4b3e-898b-bfdd13a155b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.157:15021/healthz/ready\": dial tcp 10.133.0.157:15021: connect: connection refused"
Oct 31 21:53:37.981343 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:37.981305    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:53:37.983534 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:37.983513    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:53:37.988935 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:37.988653    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 21:53:38.276055 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:38.276018    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:40.042966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:40.020531    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:40.498089 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:40.497997    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:40.606755 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:40.606724    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:41.742964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:41.736084    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:43.501794 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:43.501737    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:44.362838 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:44.362803    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:45.248789 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:45.248748    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:45.714765 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:45.714724    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:46.195616 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:46.195565    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:46.513558 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:46.513522    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:47.060410 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:47.060375    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:47.329282 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:47.329193    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-c34cbd-559d4ccccf-bxss4"
Oct 31 21:53:47.330193 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:47.330165    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c34cbd-7dc696c947-v5wpt"
Oct 31 21:53:47.686830 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:47.685122    4460 patch_prober.go:29] interesting pod/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.160:15021/healthz/ready\": read tcp 10.133.0.2:52624->10.133.0.160:15021: read: connection reset by peer" start-of-body=
Oct 31 21:53:47.686830 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:47.685203    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf" podUID="b07cb206-7fc8-42a4-9086-dac5ce1d7019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.160:15021/healthz/ready\": read tcp 10.133.0.2:52624->10.133.0.160:15021: read: connection reset by peer"
Oct 31 21:53:48.167713 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:48.167673    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:48.496499 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:48.496407    4460 generic.go:356] "Generic (PLEG): container finished" podID="b07cb206-7fc8-42a4-9086-dac5ce1d7019" containerID="03dcb7cdafbd70b4db5bada1f9d3ca6ab727fec4ccd5325598f97683283b6967" exitCode=0
Oct 31 21:53:48.496499 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:48.496477    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf" event={"ID":"b07cb206-7fc8-42a4-9086-dac5ce1d7019","Type":"ContainerDied","Data":"03dcb7cdafbd70b4db5bada1f9d3ca6ab727fec4ccd5325598f97683283b6967"}
Oct 31 21:53:48.497026 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:48.497007    4460 scope.go:122] "RemoveContainer" containerID="03dcb7cdafbd70b4db5bada1f9d3ca6ab727fec4ccd5325598f97683283b6967"
Oct 31 21:53:48.498713 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:48.498685    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:53:48.498808 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:48.498756    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:53:48.498808 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:48.498794    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:53:48.882223 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:48.882187    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:49.505774 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:49.505738    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf" event={"ID":"b07cb206-7fc8-42a4-9086-dac5ce1d7019","Type":"ContainerStarted","Data":"4f67c0688eda6f578a35f1cfa5c5fc9795bfa35c473f78180c0933a13418ebf3"}
Oct 31 21:53:49.892139 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:49.892102    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:50.003762 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:50.003724    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:50.443619 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:50.443583    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:53:50.444683 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:50.443917    4460 patch_prober.go:29] interesting pod/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused" start-of-body=
Oct 31 21:53:50.444897 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:50.444871    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf" podUID="b07cb206-7fc8-42a4-9086-dac5ce1d7019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused"
Oct 31 21:53:50.845716 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:50.845674    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:51.429597 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:51.429558    4460 patch_prober.go:29] interesting pod/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused" start-of-body=
Oct 31 21:53:51.430206 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:51.430170    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf" podUID="b07cb206-7fc8-42a4-9086-dac5ce1d7019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused"
Oct 31 21:53:51.473233 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:53:51.473186    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:53:51.613237 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:51.613203    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:51.746416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:51.746298    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:51.798326 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:51.798294    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-56a6d4-7769777f8c-2jddx"]
Oct 31 21:53:51.802760 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:51.802740    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-56a6d4-7769777f8c-2jddx"
Oct 31 21:53:51.850873 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:51.850495    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-56a6d4\""
Oct 31 21:53:51.850873 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:51.850637    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-56a6d4-dockercfg-fsf8h\""
Oct 31 21:53:51.854450 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:51.854417    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-56a6d4-7769777f8c-2jddx"]
Oct 31 21:53:51.951982 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:51.948562    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjqq5\" (UniqueName: \"kubernetes.io/projected/4648d201-9e87-495c-b4aa-cb961fae6f26-kube-api-access-qjqq5\") pod \"payload-pre-processing-e2e-extmod-b-56a6d4-7769777f8c-2jddx\" (UID: \"4648d201-9e87-495c-b4aa-cb961fae6f26\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-56a6d4-7769777f8c-2jddx"
Oct 31 21:53:51.951982 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:51.948732    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4648d201-9e87-495c-b4aa-cb961fae6f26-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-56a6d4-7769777f8c-2jddx\" (UID: \"4648d201-9e87-495c-b4aa-cb961fae6f26\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-56a6d4-7769777f8c-2jddx"
Oct 31 21:53:51.987117 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:51.982408    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-56a6d4-6758676965-4l9dm"]
Oct 31 21:53:52.046183 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:52.031319    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-56a6d4-6758676965-4l9dm"
Oct 31 21:53:52.070137 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:52.070094    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjqq5\" (UniqueName: \"kubernetes.io/projected/4648d201-9e87-495c-b4aa-cb961fae6f26-kube-api-access-qjqq5\") pod \"payload-pre-processing-e2e-extmod-b-56a6d4-7769777f8c-2jddx\" (UID: \"4648d201-9e87-495c-b4aa-cb961fae6f26\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-56a6d4-7769777f8c-2jddx"
Oct 31 21:53:52.083968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:52.070587    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4648d201-9e87-495c-b4aa-cb961fae6f26-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-56a6d4-7769777f8c-2jddx\" (UID: \"4648d201-9e87-495c-b4aa-cb961fae6f26\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-56a6d4-7769777f8c-2jddx"
Oct 31 21:53:52.084529 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:52.084494    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4648d201-9e87-495c-b4aa-cb961fae6f26-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-56a6d4-7769777f8c-2jddx\" (UID: \"4648d201-9e87-495c-b4aa-cb961fae6f26\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-56a6d4-7769777f8c-2jddx"
Oct 31 21:53:52.104965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:52.088808    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-56a6d4-6758676965-4l9dm"]
Oct 31 21:53:52.155963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:52.155109    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjqq5\" (UniqueName: \"kubernetes.io/projected/4648d201-9e87-495c-b4aa-cb961fae6f26-kube-api-access-qjqq5\") pod \"payload-pre-processing-e2e-extmod-b-56a6d4-7769777f8c-2jddx\" (UID: \"4648d201-9e87-495c-b4aa-cb961fae6f26\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-56a6d4-7769777f8c-2jddx"
Oct 31 21:53:52.186088 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:52.186048    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7cwt\" (UniqueName: \"kubernetes.io/projected/62d79c06-55b1-4b09-aa24-dd5519154921-kube-api-access-h7cwt\") pod \"payload-processing-e2e-extmod-b-56a6d4-6758676965-4l9dm\" (UID: \"62d79c06-55b1-4b09-aa24-dd5519154921\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-56a6d4-6758676965-4l9dm"
Oct 31 21:53:52.186259 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:52.186179    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62d79c06-55b1-4b09-aa24-dd5519154921-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-56a6d4-6758676965-4l9dm\" (UID: \"62d79c06-55b1-4b09-aa24-dd5519154921\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-56a6d4-6758676965-4l9dm"
Oct 31 21:53:52.288140 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:52.288105    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62d79c06-55b1-4b09-aa24-dd5519154921-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-56a6d4-6758676965-4l9dm\" (UID: \"62d79c06-55b1-4b09-aa24-dd5519154921\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-56a6d4-6758676965-4l9dm"
Oct 31 21:53:52.288392 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:52.288374    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7cwt\" (UniqueName: \"kubernetes.io/projected/62d79c06-55b1-4b09-aa24-dd5519154921-kube-api-access-h7cwt\") pod \"payload-processing-e2e-extmod-b-56a6d4-6758676965-4l9dm\" (UID: \"62d79c06-55b1-4b09-aa24-dd5519154921\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-56a6d4-6758676965-4l9dm"
Oct 31 21:53:52.288875 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:52.288852    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62d79c06-55b1-4b09-aa24-dd5519154921-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-56a6d4-6758676965-4l9dm\" (UID: \"62d79c06-55b1-4b09-aa24-dd5519154921\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-56a6d4-6758676965-4l9dm"
Oct 31 21:53:52.313797 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:52.313725    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7cwt\" (UniqueName: \"kubernetes.io/projected/62d79c06-55b1-4b09-aa24-dd5519154921-kube-api-access-h7cwt\") pod \"payload-processing-e2e-extmod-b-56a6d4-6758676965-4l9dm\" (UID: \"62d79c06-55b1-4b09-aa24-dd5519154921\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-56a6d4-6758676965-4l9dm"
Oct 31 21:53:52.424508 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:52.424204    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-56a6d4-7769777f8c-2jddx"
Oct 31 21:53:52.444416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:52.442199    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-56a6d4-6758676965-4l9dm"
Oct 31 21:53:52.445747 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:52.445570    4460 patch_prober.go:29] interesting pod/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused" start-of-body=
Oct 31 21:53:52.445747 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:52.445625    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf" podUID="b07cb206-7fc8-42a4-9086-dac5ce1d7019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused"
Oct 31 21:53:52.648181 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:52.648037    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:52.775587 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:52.775533    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-56a6d4-7769777f8c-2jddx"]
Oct 31 21:53:52.780658 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:53:52.780624    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4648d201_9e87_495c_b4aa_cb961fae6f26.slice/crio-270a878cce0e49bd5489c73cbaba24639a238c826b5576019b771cd494c08d9c WatchSource:0}: Error finding container 270a878cce0e49bd5489c73cbaba24639a238c826b5576019b771cd494c08d9c: Status 404 returned error can't find the container with id 270a878cce0e49bd5489c73cbaba24639a238c826b5576019b771cd494c08d9c
Oct 31 21:53:52.799570 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:53:52.799542    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62d79c06_55b1_4b09_aa24_dd5519154921.slice/crio-f81c12bd1387fbf5cf27ac6703ee8ff28f4d169a2c40e572d2bab8ed4a0b143b WatchSource:0}: Error finding container f81c12bd1387fbf5cf27ac6703ee8ff28f4d169a2c40e572d2bab8ed4a0b143b: Status 404 returned error can't find the container with id f81c12bd1387fbf5cf27ac6703ee8ff28f4d169a2c40e572d2bab8ed4a0b143b
Oct 31 21:53:52.799676 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:52.799659    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-56a6d4-6758676965-4l9dm"]
Oct 31 21:53:53.443022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:53.442378    4460 patch_prober.go:29] interesting pod/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused" start-of-body=
Oct 31 21:53:53.443022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:53.442444    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf" podUID="b07cb206-7fc8-42a4-9086-dac5ce1d7019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused"
Oct 31 21:53:53.566585 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:53.566547    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-56a6d4-6758676965-4l9dm" event={"ID":"62d79c06-55b1-4b09-aa24-dd5519154921","Type":"ContainerStarted","Data":"05daacd7275ef8dc3c747c0cfb4dce9f453f6b4612ea81360e181295d7ea2300"}
Oct 31 21:53:53.567128 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:53.566600    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-56a6d4-6758676965-4l9dm" event={"ID":"62d79c06-55b1-4b09-aa24-dd5519154921","Type":"ContainerStarted","Data":"f81c12bd1387fbf5cf27ac6703ee8ff28f4d169a2c40e572d2bab8ed4a0b143b"}
Oct 31 21:53:53.567128 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:53.566645    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-56a6d4-6758676965-4l9dm"
Oct 31 21:53:53.568115 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:53.568084    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-56a6d4-7769777f8c-2jddx" event={"ID":"4648d201-9e87-495c-b4aa-cb961fae6f26","Type":"ContainerStarted","Data":"8c80d3548b170139211b4a731e1face65c822d37f515e09ca4c07d0b945cbb3d"}
Oct 31 21:53:53.568248 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:53.568137    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-56a6d4-7769777f8c-2jddx" event={"ID":"4648d201-9e87-495c-b4aa-cb961fae6f26","Type":"ContainerStarted","Data":"270a878cce0e49bd5489c73cbaba24639a238c826b5576019b771cd494c08d9c"}
Oct 31 21:53:53.568248 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:53.568160    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-56a6d4-7769777f8c-2jddx"
Oct 31 21:53:53.614619 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:53.614556    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-b-56a6d4-6758676965-4l9dm" podStartSLOduration=2.614541745 podStartE2EDuration="2.614541745s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:53:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:53:53.61215924 +0000 UTC m=+2273.528468670" watchObservedRunningTime="2026-10-31 21:53:53.614541745 +0000 UTC m=+2273.530851174"
Oct 31 21:53:53.666888 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:53.666813    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-56a6d4-7769777f8c-2jddx" podStartSLOduration=2.6667957749999998 podStartE2EDuration="2.666795775s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:53:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:53:53.664273088 +0000 UTC m=+2273.580582531" watchObservedRunningTime="2026-10-31 21:53:53.666795775 +0000 UTC m=+2273.583105204"
Oct 31 21:53:54.317538 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:54.317503    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:54.430387 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:54.430350    4460 patch_prober.go:29] interesting pod/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused" start-of-body=
Oct 31 21:53:54.430694 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:54.430661    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf" podUID="b07cb206-7fc8-42a4-9086-dac5ce1d7019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused"
Oct 31 21:53:55.430698 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:55.430670    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:53:55.431158 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:55.431134    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:53:55.431890 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:55.431871    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 21:53:56.028537 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:56.028508    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:57.699982 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:57.699925    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:53:59.400324 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:53:59.400287    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:01.201820 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:01.201783    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:01.322049 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:01.322014    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:01.434318 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:01.434276    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:02.698785 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:02.698749    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:03.119659 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:03.119621    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:04.578394 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:04.578360    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-56a6d4-7769777f8c-2jddx"
Oct 31 21:54:04.578899 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:04.578795    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-56a6d4-6758676965-4l9dm"
Oct 31 21:54:04.843618 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:04.843532    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:05.319531 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:05.319496    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:54:05.955443 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:05.955410    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58"]
Oct 31 21:54:06.004071 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:06.004033    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:06.649346 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:06.649313    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:07.554362 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:07.554326    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:08.095784 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:08.095745    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:08.379574 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:08.379481    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"]
Oct 31 21:54:08.380088 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:08.380051    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m" podUID="082906e8-2fd4-4fdf-95a2-40a2e433982b" containerName="istio-proxy" containerID="cri-o://3bc0bd7f8ba7588730430797c9d791203f359fa44dcc4e7d2f41d87b90fe5cb6" gracePeriod=30
Oct 31 21:54:08.486090 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:08.486052    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:08.608283 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:08.608244    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:09.759881 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:09.759840    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"]
Oct 31 21:54:09.760374 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:09.760087    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b" podUID="076f0db9-6f89-46d7-ad9c-9855f6dbf911" containerName="istio-proxy" containerID="cri-o://eabf64413d846d3573428a026d107ae35cb064c113f7673be654fe35fc12a26c" gracePeriod=30
Oct 31 21:54:10.457711 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:10.457660    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:12.299988 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:12.299919    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:12.437373 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:12.437334    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:12.579821 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:12.579724    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:13.677629 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.677600    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:54:13.764833 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.764788    4460 generic.go:356] "Generic (PLEG): container finished" podID="082906e8-2fd4-4fdf-95a2-40a2e433982b" containerID="3bc0bd7f8ba7588730430797c9d791203f359fa44dcc4e7d2f41d87b90fe5cb6" exitCode=0
Oct 31 21:54:13.765057 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.764837    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"
Oct 31 21:54:13.765057 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.764863    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m" event={"ID":"082906e8-2fd4-4fdf-95a2-40a2e433982b","Type":"ContainerDied","Data":"3bc0bd7f8ba7588730430797c9d791203f359fa44dcc4e7d2f41d87b90fe5cb6"}
Oct 31 21:54:13.765057 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.764927    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m" event={"ID":"082906e8-2fd4-4fdf-95a2-40a2e433982b","Type":"ContainerDied","Data":"9c3701f0c5b31483bf19ca9a1d60adad5862aa0c993e74c13f8a539cce84ad5a"}
Oct 31 21:54:13.765057 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.765009    4460 scope.go:122] "RemoveContainer" containerID="3bc0bd7f8ba7588730430797c9d791203f359fa44dcc4e7d2f41d87b90fe5cb6"
Oct 31 21:54:13.778878 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.778850    4460 scope.go:122] "RemoveContainer" containerID="3bc0bd7f8ba7588730430797c9d791203f359fa44dcc4e7d2f41d87b90fe5cb6"
Oct 31 21:54:13.779379 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:54:13.779346    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bc0bd7f8ba7588730430797c9d791203f359fa44dcc4e7d2f41d87b90fe5cb6\": container with ID starting with 3bc0bd7f8ba7588730430797c9d791203f359fa44dcc4e7d2f41d87b90fe5cb6 not found: ID does not exist" containerID="3bc0bd7f8ba7588730430797c9d791203f359fa44dcc4e7d2f41d87b90fe5cb6"
Oct 31 21:54:13.779534 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.779391    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bc0bd7f8ba7588730430797c9d791203f359fa44dcc4e7d2f41d87b90fe5cb6"} err="failed to get container status \"3bc0bd7f8ba7588730430797c9d791203f359fa44dcc4e7d2f41d87b90fe5cb6\": rpc error: code = NotFound desc = could not find container \"3bc0bd7f8ba7588730430797c9d791203f359fa44dcc4e7d2f41d87b90fe5cb6\": container with ID starting with 3bc0bd7f8ba7588730430797c9d791203f359fa44dcc4e7d2f41d87b90fe5cb6 not found: ID does not exist"
Oct 31 21:54:13.815388 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.815354    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-workload-certs\") pod \"082906e8-2fd4-4fdf-95a2-40a2e433982b\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") "
Oct 31 21:54:13.815588 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.815436    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-credential-socket\") pod \"082906e8-2fd4-4fdf-95a2-40a2e433982b\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") "
Oct 31 21:54:13.815588 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.815463    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-istio-envoy\") pod \"082906e8-2fd4-4fdf-95a2-40a2e433982b\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") "
Oct 31 21:54:13.815588 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.815487    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/082906e8-2fd4-4fdf-95a2-40a2e433982b-kube-api-access-7hwhr\" (UniqueName: \"kubernetes.io/projected/082906e8-2fd4-4fdf-95a2-40a2e433982b-kube-api-access-7hwhr\") pod \"082906e8-2fd4-4fdf-95a2-40a2e433982b\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") "
Oct 31 21:54:13.815588 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.815528    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/082906e8-2fd4-4fdf-95a2-40a2e433982b-istio-token\" (UniqueName: \"kubernetes.io/projected/082906e8-2fd4-4fdf-95a2-40a2e433982b-istio-token\") pod \"082906e8-2fd4-4fdf-95a2-40a2e433982b\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") "
Oct 31 21:54:13.815588 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.815575    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-workload-socket\") pod \"082906e8-2fd4-4fdf-95a2-40a2e433982b\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") "
Oct 31 21:54:13.815876 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.815653    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/082906e8-2fd4-4fdf-95a2-40a2e433982b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/082906e8-2fd4-4fdf-95a2-40a2e433982b-istiod-ca-cert\") pod \"082906e8-2fd4-4fdf-95a2-40a2e433982b\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") "
Oct 31 21:54:13.815876 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.815685    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-workload-certs" pod "082906e8-2fd4-4fdf-95a2-40a2e433982b" (UID: "082906e8-2fd4-4fdf-95a2-40a2e433982b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:54:13.815876 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.815818    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-istio-data\") pod \"082906e8-2fd4-4fdf-95a2-40a2e433982b\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") "
Oct 31 21:54:13.815876 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.815825    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-credential-socket" pod "082906e8-2fd4-4fdf-95a2-40a2e433982b" (UID: "082906e8-2fd4-4fdf-95a2-40a2e433982b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:54:13.815876 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.815869    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/082906e8-2fd4-4fdf-95a2-40a2e433982b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/082906e8-2fd4-4fdf-95a2-40a2e433982b-istio-podinfo\") pod \"082906e8-2fd4-4fdf-95a2-40a2e433982b\" (UID: \"082906e8-2fd4-4fdf-95a2-40a2e433982b\") "
Oct 31 21:54:13.816194 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.816123    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-workload-socket" pod "082906e8-2fd4-4fdf-95a2-40a2e433982b" (UID: "082906e8-2fd4-4fdf-95a2-40a2e433982b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:54:13.816194 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.816145    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/082906e8-2fd4-4fdf-95a2-40a2e433982b-istiod-ca-cert" pod "082906e8-2fd4-4fdf-95a2-40a2e433982b" (UID: "082906e8-2fd4-4fdf-95a2-40a2e433982b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:54:13.816315 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.816192    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-istio-data" pod "082906e8-2fd4-4fdf-95a2-40a2e433982b" (UID: "082906e8-2fd4-4fdf-95a2-40a2e433982b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:54:13.816625 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.816604    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:54:13.816625 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.816624    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:54:13.816766 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.816634    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:54:13.816766 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.816648    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:54:13.816766 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.816657    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/082906e8-2fd4-4fdf-95a2-40a2e433982b-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:54:13.818034 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.817999    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/082906e8-2fd4-4fdf-95a2-40a2e433982b-istio-podinfo" pod "082906e8-2fd4-4fdf-95a2-40a2e433982b" (UID: "082906e8-2fd4-4fdf-95a2-40a2e433982b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:54:13.818156 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.818076    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/082906e8-2fd4-4fdf-95a2-40a2e433982b-kube-api-access-7hwhr" pod "082906e8-2fd4-4fdf-95a2-40a2e433982b" (UID: "082906e8-2fd4-4fdf-95a2-40a2e433982b"). InnerVolumeSpecName "kube-api-access-7hwhr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:54:13.818253 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.818233    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-istio-envoy" pod "082906e8-2fd4-4fdf-95a2-40a2e433982b" (UID: "082906e8-2fd4-4fdf-95a2-40a2e433982b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:54:13.818312 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.818265    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/082906e8-2fd4-4fdf-95a2-40a2e433982b-istio-token" pod "082906e8-2fd4-4fdf-95a2-40a2e433982b" (UID: "082906e8-2fd4-4fdf-95a2-40a2e433982b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:54:13.918459 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.918359    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/082906e8-2fd4-4fdf-95a2-40a2e433982b-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:54:13.918459 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.918397    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hwhr\" (UniqueName: \"kubernetes.io/projected/082906e8-2fd4-4fdf-95a2-40a2e433982b-kube-api-access-7hwhr\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:54:13.918459 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.918406    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/082906e8-2fd4-4fdf-95a2-40a2e433982b-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:54:13.918459 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:13.918418    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/082906e8-2fd4-4fdf-95a2-40a2e433982b-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:54:14.106478 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:14.106425    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:14.125512 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:14.125477    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"]
Oct 31 21:54:14.139560 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:14.139514    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-4499m"]
Oct 31 21:54:14.220772 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:14.220664    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:14.227166 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:14.227133    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="082906e8-2fd4-4fdf-95a2-40a2e433982b" path="/var/lib/kubelet/pods/082906e8-2fd4-4fdf-95a2-40a2e433982b/volumes"
Oct 31 21:54:14.272770 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:14.272724    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-696f9a-fc885d6c9-pnrxl"]
Oct 31 21:54:14.273567 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:14.273544    4460 cpu_manager.go:414] "containerMap: removing container" podUID="082906e8-2fd4-4fdf-95a2-40a2e433982b" containerName="istio-proxy"
Oct 31 21:54:14.273567 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:14.273566    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="082906e8-2fd4-4fdf-95a2-40a2e433982b" containerName="istio-proxy"
Oct 31 21:54:14.273750 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:14.273702    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="082906e8-2fd4-4fdf-95a2-40a2e433982b" containerName="istio-proxy"
Oct 31 21:54:14.279141 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:14.279113    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-696f9a-fc885d6c9-pnrxl"
Oct 31 21:54:14.297003 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:14.296961    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-696f9a-fc885d6c9-pnrxl"]
Oct 31 21:54:14.297826 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:14.297792    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-696f9a\""
Oct 31 21:54:14.298021 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:14.297909    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-696f9a-dockercfg-sgwsz\""
Oct 31 21:54:14.426964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:14.426014    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxmbl\" (UniqueName: \"kubernetes.io/projected/de079877-5ceb-4370-b8f6-9f6644de83b3-kube-api-access-mxmbl\") pod \"payload-pre-processing-e2e-extmod-a-696f9a-fc885d6c9-pnrxl\" (UID: \"de079877-5ceb-4370-b8f6-9f6644de83b3\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-696f9a-fc885d6c9-pnrxl"
Oct 31 21:54:14.426964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:14.426088    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de079877-5ceb-4370-b8f6-9f6644de83b3-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-696f9a-fc885d6c9-pnrxl\" (UID: \"de079877-5ceb-4370-b8f6-9f6644de83b3\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-696f9a-fc885d6c9-pnrxl"
Oct 31 21:54:14.484068 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:14.480086    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-696f9a-67f544df69-gb7nm"]
Oct 31 21:54:14.565205 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:14.550840    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de079877-5ceb-4370-b8f6-9f6644de83b3-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-696f9a-fc885d6c9-pnrxl\" (UID: \"de079877-5ceb-4370-b8f6-9f6644de83b3\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-696f9a-fc885d6c9-pnrxl"
Oct 31 21:54:14.578068 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:14.576374    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxmbl\" (UniqueName: \"kubernetes.io/projected/de079877-5ceb-4370-b8f6-9f6644de83b3-kube-api-access-mxmbl\") pod \"payload-pre-processing-e2e-extmod-a-696f9a-fc885d6c9-pnrxl\" (UID: \"de079877-5ceb-4370-b8f6-9f6644de83b3\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-696f9a-fc885d6c9-pnrxl"
Oct 31 21:54:14.578068 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:14.554747    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de079877-5ceb-4370-b8f6-9f6644de83b3-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-696f9a-fc885d6c9-pnrxl\" (UID: \"de079877-5ceb-4370-b8f6-9f6644de83b3\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-696f9a-fc885d6c9-pnrxl"
Oct 31 21:54:14.584377 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:14.583295    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-696f9a-67f544df69-gb7nm"]
Oct 31 21:54:14.584377 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:14.583430    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-696f9a-67f544df69-gb7nm"
Oct 31 21:54:14.602966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:14.602109    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:14.612095 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:14.611872    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxmbl\" (UniqueName: \"kubernetes.io/projected/de079877-5ceb-4370-b8f6-9f6644de83b3-kube-api-access-mxmbl\") pod \"payload-pre-processing-e2e-extmod-a-696f9a-fc885d6c9-pnrxl\" (UID: \"de079877-5ceb-4370-b8f6-9f6644de83b3\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-696f9a-fc885d6c9-pnrxl"
Oct 31 21:54:14.678977 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:14.677496    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/869e1829-ba34-4ed7-bd97-37aa003a0d0f-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-696f9a-67f544df69-gb7nm\" (UID: \"869e1829-ba34-4ed7-bd97-37aa003a0d0f\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-696f9a-67f544df69-gb7nm"
Oct 31 21:54:14.678977 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:14.677690    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwwtz\" (UniqueName: \"kubernetes.io/projected/869e1829-ba34-4ed7-bd97-37aa003a0d0f-kube-api-access-mwwtz\") pod \"payload-processing-e2e-extmod-a-696f9a-67f544df69-gb7nm\" (UID: \"869e1829-ba34-4ed7-bd97-37aa003a0d0f\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-696f9a-67f544df69-gb7nm"
Oct 31 21:54:14.778984 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:14.778895    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwwtz\" (UniqueName: \"kubernetes.io/projected/869e1829-ba34-4ed7-bd97-37aa003a0d0f-kube-api-access-mwwtz\") pod \"payload-processing-e2e-extmod-a-696f9a-67f544df69-gb7nm\" (UID: \"869e1829-ba34-4ed7-bd97-37aa003a0d0f\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-696f9a-67f544df69-gb7nm"
Oct 31 21:54:14.779164 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:14.779117    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/869e1829-ba34-4ed7-bd97-37aa003a0d0f-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-696f9a-67f544df69-gb7nm\" (UID: \"869e1829-ba34-4ed7-bd97-37aa003a0d0f\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-696f9a-67f544df69-gb7nm"
Oct 31 21:54:14.779833 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:14.779814    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/869e1829-ba34-4ed7-bd97-37aa003a0d0f-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-696f9a-67f544df69-gb7nm\" (UID: \"869e1829-ba34-4ed7-bd97-37aa003a0d0f\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-696f9a-67f544df69-gb7nm"
Oct 31 21:54:14.805057 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:14.805017    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwwtz\" (UniqueName: \"kubernetes.io/projected/869e1829-ba34-4ed7-bd97-37aa003a0d0f-kube-api-access-mwwtz\") pod \"payload-processing-e2e-extmod-a-696f9a-67f544df69-gb7nm\" (UID: \"869e1829-ba34-4ed7-bd97-37aa003a0d0f\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-696f9a-67f544df69-gb7nm"
Oct 31 21:54:14.899277 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:14.899233    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-696f9a-fc885d6c9-pnrxl"
Oct 31 21:54:14.946934 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:14.946895    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-696f9a-67f544df69-gb7nm"
Oct 31 21:54:15.058524 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.058152    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:54:15.086166 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.086122    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istio-podinfo\") pod \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") "
Oct 31 21:54:15.086378 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.086190    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/076f0db9-6f89-46d7-ad9c-9855f6dbf911-kube-api-access-gjmxp\" (UniqueName: \"kubernetes.io/projected/076f0db9-6f89-46d7-ad9c-9855f6dbf911-kube-api-access-gjmxp\") pod \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") "
Oct 31 21:54:15.086378 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.086241    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-workload-certs\") pod \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") "
Oct 31 21:54:15.086378 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.086267    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-workload-socket\") pod \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") "
Oct 31 21:54:15.086594 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.086426    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-credential-socket\") pod \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") "
Oct 31 21:54:15.086594 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.086579    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istiod-ca-cert\") pod \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") "
Oct 31 21:54:15.086721 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.086626    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istio-envoy\") pod \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") "
Oct 31 21:54:15.086721 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.086662    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istio-token\" (UniqueName: \"kubernetes.io/projected/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istio-token\") pod \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") "
Oct 31 21:54:15.086842 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.086741    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istio-data\") pod \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\" (UID: \"076f0db9-6f89-46d7-ad9c-9855f6dbf911\") "
Oct 31 21:54:15.086903 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.086848    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-workload-certs" pod "076f0db9-6f89-46d7-ad9c-9855f6dbf911" (UID: "076f0db9-6f89-46d7-ad9c-9855f6dbf911"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:54:15.087671 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.087370    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istiod-ca-cert" pod "076f0db9-6f89-46d7-ad9c-9855f6dbf911" (UID: "076f0db9-6f89-46d7-ad9c-9855f6dbf911"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:54:15.087671 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.087424    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:54:15.087671 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.087623    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istio-data" pod "076f0db9-6f89-46d7-ad9c-9855f6dbf911" (UID: "076f0db9-6f89-46d7-ad9c-9855f6dbf911"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:54:15.088207 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.088027    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-workload-socket" pod "076f0db9-6f89-46d7-ad9c-9855f6dbf911" (UID: "076f0db9-6f89-46d7-ad9c-9855f6dbf911"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:54:15.088388 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.088357    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-credential-socket" pod "076f0db9-6f89-46d7-ad9c-9855f6dbf911" (UID: "076f0db9-6f89-46d7-ad9c-9855f6dbf911"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:54:15.090746 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.090603    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istio-envoy" pod "076f0db9-6f89-46d7-ad9c-9855f6dbf911" (UID: "076f0db9-6f89-46d7-ad9c-9855f6dbf911"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:54:15.090746 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.091232    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istio-podinfo" pod "076f0db9-6f89-46d7-ad9c-9855f6dbf911" (UID: "076f0db9-6f89-46d7-ad9c-9855f6dbf911"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:54:15.092478 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.092450    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istio-token" pod "076f0db9-6f89-46d7-ad9c-9855f6dbf911" (UID: "076f0db9-6f89-46d7-ad9c-9855f6dbf911"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:54:15.092784 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.092761    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/076f0db9-6f89-46d7-ad9c-9855f6dbf911-kube-api-access-gjmxp" pod "076f0db9-6f89-46d7-ad9c-9855f6dbf911" (UID: "076f0db9-6f89-46d7-ad9c-9855f6dbf911"). InnerVolumeSpecName "kube-api-access-gjmxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:54:15.188706 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.188670    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:54:15.188706 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.188702    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:54:15.188706 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.188711    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:54:15.189083 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.188719    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:54:15.189083 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.188728    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:54:15.189083 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.188735    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/076f0db9-6f89-46d7-ad9c-9855f6dbf911-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:54:15.189083 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.188743    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjmxp\" (UniqueName: \"kubernetes.io/projected/076f0db9-6f89-46d7-ad9c-9855f6dbf911-kube-api-access-gjmxp\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:54:15.189083 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.188752    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/076f0db9-6f89-46d7-ad9c-9855f6dbf911-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:54:15.225189 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.225158    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-696f9a-fc885d6c9-pnrxl"]
Oct 31 21:54:15.226064 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:54:15.226037    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde079877_5ceb_4370_b8f6_9f6644de83b3.slice/crio-80dafd19e1fea0ec25e0e80c17199054fb634ff04ec38f9f029c2a57d393af63 WatchSource:0}: Error finding container 80dafd19e1fea0ec25e0e80c17199054fb634ff04ec38f9f029c2a57d393af63: Status 404 returned error can't find the container with id 80dafd19e1fea0ec25e0e80c17199054fb634ff04ec38f9f029c2a57d393af63
Oct 31 21:54:15.284931 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.284861    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-696f9a-67f544df69-gb7nm"]
Oct 31 21:54:15.303674 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:54:15.303636    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod869e1829_ba34_4ed7_bd97_37aa003a0d0f.slice/crio-0898fdfca23d8faafad7eb7e2dc8d49ccbead516250fa69da50e9230383ed389 WatchSource:0}: Error finding container 0898fdfca23d8faafad7eb7e2dc8d49ccbead516250fa69da50e9230383ed389: Status 404 returned error can't find the container with id 0898fdfca23d8faafad7eb7e2dc8d49ccbead516250fa69da50e9230383ed389
Oct 31 21:54:15.783646 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.783602    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-696f9a-fc885d6c9-pnrxl" event={"ID":"de079877-5ceb-4370-b8f6-9f6644de83b3","Type":"ContainerStarted","Data":"aad578a8ba1d909be721ccd1999fd710e09e470e5f24ad2bb8035af516118252"}
Oct 31 21:54:15.784124 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.783673    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-696f9a-fc885d6c9-pnrxl" event={"ID":"de079877-5ceb-4370-b8f6-9f6644de83b3","Type":"ContainerStarted","Data":"80dafd19e1fea0ec25e0e80c17199054fb634ff04ec38f9f029c2a57d393af63"}
Oct 31 21:54:15.784124 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.783701    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-696f9a-fc885d6c9-pnrxl"
Oct 31 21:54:15.785065 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.785039    4460 generic.go:356] "Generic (PLEG): container finished" podID="076f0db9-6f89-46d7-ad9c-9855f6dbf911" containerID="eabf64413d846d3573428a026d107ae35cb064c113f7673be654fe35fc12a26c" exitCode=0
Oct 31 21:54:15.785201 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.785094    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"
Oct 31 21:54:15.785201 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.785122    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b" event={"ID":"076f0db9-6f89-46d7-ad9c-9855f6dbf911","Type":"ContainerDied","Data":"eabf64413d846d3573428a026d107ae35cb064c113f7673be654fe35fc12a26c"}
Oct 31 21:54:15.785201 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.785178    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b" event={"ID":"076f0db9-6f89-46d7-ad9c-9855f6dbf911","Type":"ContainerDied","Data":"a953f891cf4ec0a2ed798fe944369f137c033553b01a297ba11a020694e6831f"}
Oct 31 21:54:15.785201 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.785201    4460 scope.go:122] "RemoveContainer" containerID="eabf64413d846d3573428a026d107ae35cb064c113f7673be654fe35fc12a26c"
Oct 31 21:54:15.787464 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.787434    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-696f9a-67f544df69-gb7nm" event={"ID":"869e1829-ba34-4ed7-bd97-37aa003a0d0f","Type":"ContainerStarted","Data":"6abf9618c0148ceb6158fe31ae2e799e97136ead6f17693dd0fb2de1ca58fa70"}
Oct 31 21:54:15.787585 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.787496    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-696f9a-67f544df69-gb7nm" event={"ID":"869e1829-ba34-4ed7-bd97-37aa003a0d0f","Type":"ContainerStarted","Data":"0898fdfca23d8faafad7eb7e2dc8d49ccbead516250fa69da50e9230383ed389"}
Oct 31 21:54:15.787585 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.787530    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-696f9a-67f544df69-gb7nm"
Oct 31 21:54:15.798076 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.798054    4460 scope.go:122] "RemoveContainer" containerID="eabf64413d846d3573428a026d107ae35cb064c113f7673be654fe35fc12a26c"
Oct 31 21:54:15.798449 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:54:15.798423    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eabf64413d846d3573428a026d107ae35cb064c113f7673be654fe35fc12a26c\": container with ID starting with eabf64413d846d3573428a026d107ae35cb064c113f7673be654fe35fc12a26c not found: ID does not exist" containerID="eabf64413d846d3573428a026d107ae35cb064c113f7673be654fe35fc12a26c"
Oct 31 21:54:15.798527 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.798460    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eabf64413d846d3573428a026d107ae35cb064c113f7673be654fe35fc12a26c"} err="failed to get container status \"eabf64413d846d3573428a026d107ae35cb064c113f7673be654fe35fc12a26c\": rpc error: code = NotFound desc = could not find container \"eabf64413d846d3573428a026d107ae35cb064c113f7673be654fe35fc12a26c\": container with ID starting with eabf64413d846d3573428a026d107ae35cb064c113f7673be654fe35fc12a26c not found: ID does not exist"
Oct 31 21:54:15.836189 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.836129    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-696f9a-fc885d6c9-pnrxl" podStartSLOduration=1.8361137250000001 podStartE2EDuration="1.836113725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:54:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:54:15.833618537 +0000 UTC m=+2295.749927981" watchObservedRunningTime="2026-10-31 21:54:15.836113725 +0000 UTC m=+2295.752423154"
Oct 31 21:54:15.878202 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.878097    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"]
Oct 31 21:54:15.891595 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.891560    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-njj9b"]
Oct 31 21:54:15.925478 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:15.925415    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-696f9a-67f544df69-gb7nm" podStartSLOduration=1.925400295 podStartE2EDuration="1.925400295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:54:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:54:15.922614645 +0000 UTC m=+2295.838924083" watchObservedRunningTime="2026-10-31 21:54:15.925400295 +0000 UTC m=+2295.841709725"
Oct 31 21:54:16.226504 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:16.226415    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="076f0db9-6f89-46d7-ad9c-9855f6dbf911" path="/var/lib/kubelet/pods/076f0db9-6f89-46d7-ad9c-9855f6dbf911/volumes"
Oct 31 21:54:16.340008 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:16.339968    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:18.034686 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:18.034650    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:19.764392 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:19.764356    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:19.888468 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:19.888434    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:20.023342 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:20.023261    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:20.980780 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:20.980715    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58" podUID="aed4a2f6-db43-4088-a694-b38716d998aa" containerName="main" containerID="cri-o://4ea5e31fe67f03ce4f12dfb2a801aabdc316fbf3f32b59e37eae88410c25a5e2" gracePeriod=45
Oct 31 21:54:21.276870 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:21.276843    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58"
Oct 31 21:54:21.416042 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:21.416006    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aed4a2f6-db43-4088-a694-b38716d998aa-tls-certs\" (UniqueName: \"kubernetes.io/secret/aed4a2f6-db43-4088-a694-b38716d998aa-tls-certs\") pod \"aed4a2f6-db43-4088-a694-b38716d998aa\" (UID: \"aed4a2f6-db43-4088-a694-b38716d998aa\") "
Oct 31 21:54:21.416198 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:21.416053    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aed4a2f6-db43-4088-a694-b38716d998aa-kube-api-access-xqhwx\" (UniqueName: \"kubernetes.io/projected/aed4a2f6-db43-4088-a694-b38716d998aa-kube-api-access-xqhwx\") pod \"aed4a2f6-db43-4088-a694-b38716d998aa\" (UID: \"aed4a2f6-db43-4088-a694-b38716d998aa\") "
Oct 31 21:54:21.416198 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:21.416093    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aed4a2f6-db43-4088-a694-b38716d998aa-dshm\" (UniqueName: \"kubernetes.io/empty-dir/aed4a2f6-db43-4088-a694-b38716d998aa-dshm\") pod \"aed4a2f6-db43-4088-a694-b38716d998aa\" (UID: \"aed4a2f6-db43-4088-a694-b38716d998aa\") "
Oct 31 21:54:21.416198 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:21.416194    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aed4a2f6-db43-4088-a694-b38716d998aa-home\" (UniqueName: \"kubernetes.io/empty-dir/aed4a2f6-db43-4088-a694-b38716d998aa-home\") pod \"aed4a2f6-db43-4088-a694-b38716d998aa\" (UID: \"aed4a2f6-db43-4088-a694-b38716d998aa\") "
Oct 31 21:54:21.416397 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:21.416223    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aed4a2f6-db43-4088-a694-b38716d998aa-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aed4a2f6-db43-4088-a694-b38716d998aa-tmp-dir\") pod \"aed4a2f6-db43-4088-a694-b38716d998aa\" (UID: \"aed4a2f6-db43-4088-a694-b38716d998aa\") "
Oct 31 21:54:21.416397 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:21.416244    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aed4a2f6-db43-4088-a694-b38716d998aa-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aed4a2f6-db43-4088-a694-b38716d998aa-model-cache\") pod \"aed4a2f6-db43-4088-a694-b38716d998aa\" (UID: \"aed4a2f6-db43-4088-a694-b38716d998aa\") "
Oct 31 21:54:21.416522 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:21.416471    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aed4a2f6-db43-4088-a694-b38716d998aa-home" pod "aed4a2f6-db43-4088-a694-b38716d998aa" (UID: "aed4a2f6-db43-4088-a694-b38716d998aa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:54:21.416593 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:21.416500    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aed4a2f6-db43-4088-a694-b38716d998aa-tmp-dir" pod "aed4a2f6-db43-4088-a694-b38716d998aa" (UID: "aed4a2f6-db43-4088-a694-b38716d998aa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:54:21.416648 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:21.416592    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aed4a2f6-db43-4088-a694-b38716d998aa-model-cache" pod "aed4a2f6-db43-4088-a694-b38716d998aa" (UID: "aed4a2f6-db43-4088-a694-b38716d998aa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:54:21.417022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:21.417000    4460 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aed4a2f6-db43-4088-a694-b38716d998aa-home\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:54:21.417088 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:21.417053    4460 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aed4a2f6-db43-4088-a694-b38716d998aa-tmp-dir\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:54:21.417088 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:21.417071    4460 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aed4a2f6-db43-4088-a694-b38716d998aa-model-cache\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:54:21.418352 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:21.418324    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aed4a2f6-db43-4088-a694-b38716d998aa-dshm" pod "aed4a2f6-db43-4088-a694-b38716d998aa" (UID: "aed4a2f6-db43-4088-a694-b38716d998aa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:54:21.418462 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:21.418371    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aed4a2f6-db43-4088-a694-b38716d998aa-tls-certs" pod "aed4a2f6-db43-4088-a694-b38716d998aa" (UID: "aed4a2f6-db43-4088-a694-b38716d998aa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 21:54:21.418462 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:21.418379    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aed4a2f6-db43-4088-a694-b38716d998aa-kube-api-access-xqhwx" pod "aed4a2f6-db43-4088-a694-b38716d998aa" (UID: "aed4a2f6-db43-4088-a694-b38716d998aa"). InnerVolumeSpecName "kube-api-access-xqhwx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:54:21.453190 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:54:21.453157    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:54:21.517911 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:21.517886    4460 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aed4a2f6-db43-4088-a694-b38716d998aa-tls-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:54:21.517911 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:21.517912    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqhwx\" (UniqueName: \"kubernetes.io/projected/aed4a2f6-db43-4088-a694-b38716d998aa-kube-api-access-xqhwx\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:54:21.518145 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:21.517922    4460 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aed4a2f6-db43-4088-a694-b38716d998aa-dshm\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:54:21.543210 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:21.543151    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:21.850559 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:21.850474    4460 generic.go:356] "Generic (PLEG): container finished" podID="aed4a2f6-db43-4088-a694-b38716d998aa" containerID="4ea5e31fe67f03ce4f12dfb2a801aabdc316fbf3f32b59e37eae88410c25a5e2" exitCode=0
Oct 31 21:54:21.850711 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:21.850573    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58"
Oct 31 21:54:21.850711 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:21.850597    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58" event={"ID":"aed4a2f6-db43-4088-a694-b38716d998aa","Type":"ContainerDied","Data":"4ea5e31fe67f03ce4f12dfb2a801aabdc316fbf3f32b59e37eae88410c25a5e2"}
Oct 31 21:54:21.850711 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:21.850645    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58" event={"ID":"aed4a2f6-db43-4088-a694-b38716d998aa","Type":"ContainerDied","Data":"752e1b8ddca4c67bdc444cb549b5f549daa0d4d199d4b4539dc466eafd4fdda0"}
Oct 31 21:54:21.850711 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:21.850666    4460 scope.go:122] "RemoveContainer" containerID="4ea5e31fe67f03ce4f12dfb2a801aabdc316fbf3f32b59e37eae88410c25a5e2"
Oct 31 21:54:21.862237 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:21.862214    4460 scope.go:122] "RemoveContainer" containerID="4ea5e31fe67f03ce4f12dfb2a801aabdc316fbf3f32b59e37eae88410c25a5e2"
Oct 31 21:54:21.862582 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:54:21.862562    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ea5e31fe67f03ce4f12dfb2a801aabdc316fbf3f32b59e37eae88410c25a5e2\": container with ID starting with 4ea5e31fe67f03ce4f12dfb2a801aabdc316fbf3f32b59e37eae88410c25a5e2 not found: ID does not exist" containerID="4ea5e31fe67f03ce4f12dfb2a801aabdc316fbf3f32b59e37eae88410c25a5e2"
Oct 31 21:54:21.862710 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:21.862596    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ea5e31fe67f03ce4f12dfb2a801aabdc316fbf3f32b59e37eae88410c25a5e2"} err="failed to get container status \"4ea5e31fe67f03ce4f12dfb2a801aabdc316fbf3f32b59e37eae88410c25a5e2\": rpc error: code = NotFound desc = could not find container \"4ea5e31fe67f03ce4f12dfb2a801aabdc316fbf3f32b59e37eae88410c25a5e2\": container with ID starting with 4ea5e31fe67f03ce4f12dfb2a801aabdc316fbf3f32b59e37eae88410c25a5e2 not found: ID does not exist"
Oct 31 21:54:21.908605 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:21.908561    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58"]
Oct 31 21:54:21.921634 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:21.921610    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w1-fb32d4/auth-test-model-fb32d4-kserve-6ddc98ccdf-k7b58"]
Oct 31 21:54:22.225854 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:22.225778    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aed4a2f6-db43-4088-a694-b38716d998aa" path="/var/lib/kubelet/pods/aed4a2f6-db43-4088-a694-b38716d998aa/volumes"
Oct 31 21:54:23.323174 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:23.323137    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:25.043400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:25.043324    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:26.713038 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:26.712997    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:26.807413 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:26.807373    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-696f9a-67f544df69-gb7nm"
Oct 31 21:54:26.807614 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:26.807464    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-696f9a-fc885d6c9-pnrxl"
Oct 31 21:54:27.315601 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:27.315566    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:28.425412 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:28.425370    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:30.121731 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:30.121691    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:31.814486 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:31.814437    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:33.632301 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:33.632269    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:35.384036 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:35.384000    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:37.266312 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:37.266281    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:38.594874 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:38.594842    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:39.211923 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:39.211882    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:39.704514 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:39.704476    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:39.824714 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:39.824677    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:39.990055 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:39.989918    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:40.264638 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:40.264609    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:40.420478 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:40.420433    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:40.567883 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:40.567798    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:41.061649 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:41.061565    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:41.184350 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:41.184311    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:41.307883 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:41.307847    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:42.558618 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:42.558581    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:42.832097 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:42.832024    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5"]
Oct 31 21:54:42.833050 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:42.833024    4460 cpu_manager.go:414] "containerMap: removing container" podUID="076f0db9-6f89-46d7-ad9c-9855f6dbf911" containerName="istio-proxy"
Oct 31 21:54:42.833050 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:42.833051    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="076f0db9-6f89-46d7-ad9c-9855f6dbf911" containerName="istio-proxy"
Oct 31 21:54:42.833275 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:42.833079    4460 cpu_manager.go:414] "containerMap: removing container" podUID="aed4a2f6-db43-4088-a694-b38716d998aa" containerName="main"
Oct 31 21:54:42.833275 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:42.833085    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aed4a2f6-db43-4088-a694-b38716d998aa" containerName="main"
Oct 31 21:54:42.833275 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:42.833241    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="076f0db9-6f89-46d7-ad9c-9855f6dbf911" containerName="istio-proxy"
Oct 31 21:54:42.833275 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:42.833258    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="aed4a2f6-db43-4088-a694-b38716d998aa" containerName="main"
Oct 31 21:54:42.838232 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:42.838212    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5"
Oct 31 21:54:42.860339 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:42.860309    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-c34cbd\"/\"openshift-service-ca.crt\""
Oct 31 21:54:42.860641 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:42.860622    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-c34cbd\"/\"routing-overlay\""
Oct 31 21:54:42.860924 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:42.860904    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-c34cbd\"/\"payload-processing-external-model-plugins-e2e-extmod-b-c34cbd\""
Oct 31 21:54:42.863049 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:42.863023    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-c34cbd\"/\"kube-root-ca.crt\""
Oct 31 21:54:42.869926 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:42.869822    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:42.870365 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:42.870345    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5"]
Oct 31 21:54:42.903253 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:42.903214    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/2b649c1b-7735-4edf-8ec6-c04cc500d5c2-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5\" (UID: \"2b649c1b-7735-4edf-8ec6-c04cc500d5c2\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5"
Oct 31 21:54:42.903426 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:42.903331    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2b649c1b-7735-4edf-8ec6-c04cc500d5c2-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5\" (UID: \"2b649c1b-7735-4edf-8ec6-c04cc500d5c2\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5"
Oct 31 21:54:42.903496 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:42.903423    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b649c1b-7735-4edf-8ec6-c04cc500d5c2-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5\" (UID: \"2b649c1b-7735-4edf-8ec6-c04cc500d5c2\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5"
Oct 31 21:54:42.903625 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:42.903539    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntvtj\" (UniqueName: \"kubernetes.io/projected/2b649c1b-7735-4edf-8ec6-c04cc500d5c2-kube-api-access-ntvtj\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5\" (UID: \"2b649c1b-7735-4edf-8ec6-c04cc500d5c2\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5"
Oct 31 21:54:42.942478 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:42.942441    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-c34cbd\"/\"agc-cred-3d39da95e8af\""
Oct 31 21:54:42.942615 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:42.942585    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-c34cbd\"/\"payload-processing-external-model-e2e--bb6f7797-dockercfg-6p5zs\""
Oct 31 21:54:43.014964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:43.010381    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2b649c1b-7735-4edf-8ec6-c04cc500d5c2-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5\" (UID: \"2b649c1b-7735-4edf-8ec6-c04cc500d5c2\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5"
Oct 31 21:54:43.014964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:43.010518    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b649c1b-7735-4edf-8ec6-c04cc500d5c2-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5\" (UID: \"2b649c1b-7735-4edf-8ec6-c04cc500d5c2\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5"
Oct 31 21:54:43.014964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:43.010672    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntvtj\" (UniqueName: \"kubernetes.io/projected/2b649c1b-7735-4edf-8ec6-c04cc500d5c2-kube-api-access-ntvtj\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5\" (UID: \"2b649c1b-7735-4edf-8ec6-c04cc500d5c2\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5"
Oct 31 21:54:43.014964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:43.010777    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/2b649c1b-7735-4edf-8ec6-c04cc500d5c2-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5\" (UID: \"2b649c1b-7735-4edf-8ec6-c04cc500d5c2\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5"
Oct 31 21:54:43.027965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:43.024923    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b649c1b-7735-4edf-8ec6-c04cc500d5c2-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5\" (UID: \"2b649c1b-7735-4edf-8ec6-c04cc500d5c2\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5"
Oct 31 21:54:43.027965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:43.025584    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2b649c1b-7735-4edf-8ec6-c04cc500d5c2-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5\" (UID: \"2b649c1b-7735-4edf-8ec6-c04cc500d5c2\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5"
Oct 31 21:54:43.049962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:43.044321    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/2b649c1b-7735-4edf-8ec6-c04cc500d5c2-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5\" (UID: \"2b649c1b-7735-4edf-8ec6-c04cc500d5c2\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5"
Oct 31 21:54:43.072822 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:43.072751    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntvtj\" (UniqueName: \"kubernetes.io/projected/2b649c1b-7735-4edf-8ec6-c04cc500d5c2-kube-api-access-ntvtj\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5\" (UID: \"2b649c1b-7735-4edf-8ec6-c04cc500d5c2\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5"
Oct 31 21:54:43.164218 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:43.164120    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5"
Oct 31 21:54:43.339805 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:43.339761    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:43.497447 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:43.497361    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5"]
Oct 31 21:54:43.508423 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:54:43.508387    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b649c1b_7735_4edf_8ec6_c04cc500d5c2.slice/crio-6cb9c87357a22e3de93b4785da1b1dbc6eeb731ad6289ec10fbaa346adf33138 WatchSource:0}: Error finding container 6cb9c87357a22e3de93b4785da1b1dbc6eeb731ad6289ec10fbaa346adf33138: Status 404 returned error can't find the container with id 6cb9c87357a22e3de93b4785da1b1dbc6eeb731ad6289ec10fbaa346adf33138
Oct 31 21:54:44.086329 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:44.086288    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5" event={"ID":"2b649c1b-7735-4edf-8ec6-c04cc500d5c2","Type":"ContainerStarted","Data":"0d1fb174a012a9f6e3eba101af4fff7f301a009a13ece4b7762d73d9a0080184"}
Oct 31 21:54:44.086763 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:44.086352    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5" event={"ID":"2b649c1b-7735-4edf-8ec6-c04cc500d5c2","Type":"ContainerStarted","Data":"6cb9c87357a22e3de93b4785da1b1dbc6eeb731ad6289ec10fbaa346adf33138"}
Oct 31 21:54:44.086763 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:44.086385    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5"
Oct 31 21:54:44.135467 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:44.135415    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5" podStartSLOduration=2.135400871 podStartE2EDuration="2.135400871s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:54:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:54:44.13104963 +0000 UTC m=+2324.047359060" watchObservedRunningTime="2026-10-31 21:54:44.135400871 +0000 UTC m=+2324.051710330"
Oct 31 21:54:44.685561 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:44.685506    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:45.162119 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:45.162080    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:45.291867 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:45.291829    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:45.405380 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:45.405341    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:45.792849 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:45.792812    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:45.931068 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:45.930831    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:47.011530 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:47.011490    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:48.300291 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:48.300249    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:48.935758 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:48.935720    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:49.515714 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:49.515678    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:49.625237 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:49.625197    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:50.793487 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:50.793452    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:51.237501 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:51.237398    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:51.350232 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:51.350192    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:51.458402 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:54:51.458368    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:54:52.588304 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:52.588257    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:54.286876 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:54.286842    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:55.104118 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:55.104082    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5"
Oct 31 21:54:56.141777 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:56.141734    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:57.965351 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:57.965318    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:54:59.681428 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:54:59.681391    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:01.384115 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:01.384076    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:01.517166 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:01.517130    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:01.960754 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:01.960714    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:02.628256 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:02.628212    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:02.751986 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:02.751916    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:02.873748 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:02.873714    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:03.160830 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:03.160782    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:03.325956 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:03.325918    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:03.469014 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:03.468665    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:03.942375 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:03.942343    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:04.064277 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:04.064229    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:05.162405 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.162366    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"]
Oct 31 21:55:05.168406 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.168384    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 21:55:05.188349 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.188318    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w0-0877f5-openshift-default-dockercfg-qgncs\""
Oct 31 21:55:05.206647 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.206620    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"]
Oct 31 21:55:05.226014 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.225987    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istio-token\") pod \"e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") " pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 21:55:05.226156 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.226126    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9p4wr\" (UniqueName: \"kubernetes.io/projected/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-kube-api-access-9p4wr\") pod \"e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") " pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 21:55:05.226229 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.226171    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-workload-certs\") pod \"e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") " pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 21:55:05.226319 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.226270    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istio-podinfo\") pod \"e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") " pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 21:55:05.226388 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.226329    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istio-envoy\") pod \"e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") " pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 21:55:05.226450 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.226384    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-workload-socket\") pod \"e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") " pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 21:55:05.226516 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.226491    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-credential-socket\") pod \"e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") " pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 21:55:05.226574 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.226539    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istio-data\") pod \"e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") " pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 21:55:05.226725 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.226698    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istiod-ca-cert\") pod \"e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") " pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 21:55:05.275990 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.275954    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:05.339267 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.339224    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istio-podinfo\") pod \"e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") " pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 21:55:05.340141 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.340114    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istio-envoy\") pod \"e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") " pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 21:55:05.346997 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.344437    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istio-envoy\") pod \"e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") " pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 21:55:05.346997 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.345844    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-workload-socket\") pod \"e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") " pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 21:55:05.346997 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.346089    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-credential-socket\") pod \"e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") " pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 21:55:05.346997 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.346171    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-workload-socket\") pod \"e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") " pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 21:55:05.346997 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.346237    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istio-data\") pod \"e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") " pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 21:55:05.346997 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.346482    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istiod-ca-cert\") pod \"e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") " pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 21:55:05.346997 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.346638    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istio-token\") pod \"e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") " pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 21:55:05.346997 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.346779    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9p4wr\" (UniqueName: \"kubernetes.io/projected/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-kube-api-access-9p4wr\") pod \"e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") " pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 21:55:05.346997 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.346825    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-workload-certs\") pod \"e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") " pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 21:55:05.348453 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.348421    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istio-data\") pod \"e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") " pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 21:55:05.348594 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.348542    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-workload-certs\") pod \"e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") " pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 21:55:05.348708 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.348677    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-credential-socket\") pod \"e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") " pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 21:55:05.349355 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.349328    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istiod-ca-cert\") pod \"e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") " pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 21:55:05.349459 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.349332    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istio-podinfo\") pod \"e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") " pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 21:55:05.379635 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.379606    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istio-token\") pod \"e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") " pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 21:55:05.381883 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.381854    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9p4wr\" (UniqueName: \"kubernetes.io/projected/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-kube-api-access-9p4wr\") pod \"e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") " pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 21:55:05.476418 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.476331    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:05.483516 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.483486    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 21:55:05.763507 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.763463    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:05.810592 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.810563    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"]
Oct 31 21:55:05.819530 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.818708    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:55:05.819530 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.818852    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:55:05.819530 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:05.818958    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:55:06.245564 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:06.245284    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:06.334971 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:06.326993    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94" event={"ID":"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66","Type":"ContainerStarted","Data":"241ea9f66a5f5bc555331327dee64dc4059704670c4041a76c1e78eaecce2f79"}
Oct 31 21:55:06.335320 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:06.335285    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94" event={"ID":"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66","Type":"ContainerStarted","Data":"aedc28bc8a2500dfe1903989fbb465f641f1d28f1f5c83227dd7f4cb0512dec0"}
Oct 31 21:55:06.459793 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:06.459759    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:06.483746 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:06.483716    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 21:55:07.132544 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:07.132506    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:07.486585 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:07.486500    4460 patch_prober.go:29] interesting pod/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused" start-of-body=
Oct 31 21:55:07.486585 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:07.486564    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94" podUID="a4227bd4-a2ef-4c55-9a6a-a0347e9afa66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused"
Oct 31 21:55:08.484548 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:08.484513    4460 patch_prober.go:29] interesting pod/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused" start-of-body=
Oct 31 21:55:08.484859 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:08.484570    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94" podUID="a4227bd4-a2ef-4c55-9a6a-a0347e9afa66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused"
Oct 31 21:55:08.966616 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:08.966584    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:09.484121 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:09.484088    4460 patch_prober.go:29] interesting pod/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused" start-of-body=
Oct 31 21:55:09.484315 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:09.484139    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94" podUID="a4227bd4-a2ef-4c55-9a6a-a0347e9afa66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused"
Oct 31 21:55:09.952562 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:09.952478    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:10.070036 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:10.070002    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:10.484485 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:10.484438    4460 patch_prober.go:29] interesting pod/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused" start-of-body=
Oct 31 21:55:10.484684 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:10.484516    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94" podUID="a4227bd4-a2ef-4c55-9a6a-a0347e9afa66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused"
Oct 31 21:55:10.943377 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:10.943321    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:11.484609 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:11.484574    4460 patch_prober.go:29] interesting pod/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused" start-of-body=
Oct 31 21:55:11.485120 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:11.484633    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94" podUID="a4227bd4-a2ef-4c55-9a6a-a0347e9afa66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused"
Oct 31 21:55:11.710970 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:11.695164    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94" podStartSLOduration=6.695141549 podStartE2EDuration="6.695141549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:55:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:55:06.430414793 +0000 UTC m=+2346.346724223" watchObservedRunningTime="2026-10-31 21:55:11.695141549 +0000 UTC m=+2351.611450980"
Oct 31 21:55:11.769724 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:11.769622    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:11.774139 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:11.774114    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:55:11.892788 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:11.892751    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:12.496613 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:12.496579    4460 patch_prober.go:29] interesting pod/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused" start-of-body=
Oct 31 21:55:12.497272 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:12.497240    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94" podUID="a4227bd4-a2ef-4c55-9a6a-a0347e9afa66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.171:15021/healthz/ready\": dial tcp 10.133.0.171:15021: connect: connection refused"
Oct 31 21:55:12.749731 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:12.749608    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:13.523462 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:13.523426    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 21:55:13.523965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:13.523935    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 21:55:13.525357 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:13.525333    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 21:55:14.529188 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:14.529152    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:16.213418 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:16.213386    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:17.908763 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:17.908731    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:19.608293 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:19.608260    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:21.394181 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:21.394148    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:21.486726 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:55:21.486685    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:55:23.186502 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:23.186455    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:23.319931 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:23.319891    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:23.430034 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:23.429998    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:23.689291 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:23.689240    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:25.025408 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:25.025374    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:25.149152 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:25.149108    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:25.579044 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:25.578996    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:25.702992 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:25.702350    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:26.744127 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:26.744087    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:28.558861 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:28.558829    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:29.224804 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:29.224763    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:29.355622 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:29.355582    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:30.596044 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:30.596003    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:31.030774 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:31.030735    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:31.139049 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:31.139013    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:31.213319 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:31.213283    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-4a73b1-7f9ccbd4c9-j9ws8"]
Oct 31 21:55:31.218085 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:31.218062    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4a73b1-7f9ccbd4c9-j9ws8"
Oct 31 21:55:31.239685 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:31.239656    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-4a73b1-7f9ccbd4c9-j9ws8"]
Oct 31 21:55:31.248070 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:31.248041    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-4a73b1\""
Oct 31 21:55:31.248070 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:31.248044    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-4a73b1-dockercfg-mpqn8\""
Oct 31 21:55:31.293301 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:31.292520    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e438fbbd-062a-4ba7-9c8e-fe954d0cf55b-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-4a73b1-7f9ccbd4c9-j9ws8\" (UID: \"e438fbbd-062a-4ba7-9c8e-fe954d0cf55b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4a73b1-7f9ccbd4c9-j9ws8"
Oct 31 21:55:31.293301 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:31.292710    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flzxg\" (UniqueName: \"kubernetes.io/projected/e438fbbd-062a-4ba7-9c8e-fe954d0cf55b-kube-api-access-flzxg\") pod \"payload-pre-processing-e2e-extmod-b-4a73b1-7f9ccbd4c9-j9ws8\" (UID: \"e438fbbd-062a-4ba7-9c8e-fe954d0cf55b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4a73b1-7f9ccbd4c9-j9ws8"
Oct 31 21:55:31.308340 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:31.306526    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-4a73b1-77b884cb9-g55qb"]
Oct 31 21:55:31.354000 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:31.312674    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-4a73b1-77b884cb9-g55qb"
Oct 31 21:55:31.462002 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:31.455898    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e438fbbd-062a-4ba7-9c8e-fe954d0cf55b-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-4a73b1-7f9ccbd4c9-j9ws8\" (UID: \"e438fbbd-062a-4ba7-9c8e-fe954d0cf55b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4a73b1-7f9ccbd4c9-j9ws8"
Oct 31 21:55:31.462002 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:31.456079    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7bl4\" (UniqueName: \"kubernetes.io/projected/48f52b01-f63f-47c2-b0a5-d07370934c67-kube-api-access-z7bl4\") pod \"payload-processing-e2e-extmod-b-4a73b1-77b884cb9-g55qb\" (UID: \"48f52b01-f63f-47c2-b0a5-d07370934c67\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-4a73b1-77b884cb9-g55qb"
Oct 31 21:55:31.462002 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:31.456117    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flzxg\" (UniqueName: \"kubernetes.io/projected/e438fbbd-062a-4ba7-9c8e-fe954d0cf55b-kube-api-access-flzxg\") pod \"payload-pre-processing-e2e-extmod-b-4a73b1-7f9ccbd4c9-j9ws8\" (UID: \"e438fbbd-062a-4ba7-9c8e-fe954d0cf55b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4a73b1-7f9ccbd4c9-j9ws8"
Oct 31 21:55:31.462002 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:31.456242    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48f52b01-f63f-47c2-b0a5-d07370934c67-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-4a73b1-77b884cb9-g55qb\" (UID: \"48f52b01-f63f-47c2-b0a5-d07370934c67\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-4a73b1-77b884cb9-g55qb"
Oct 31 21:55:31.462002 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:31.457247    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e438fbbd-062a-4ba7-9c8e-fe954d0cf55b-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-4a73b1-7f9ccbd4c9-j9ws8\" (UID: \"e438fbbd-062a-4ba7-9c8e-fe954d0cf55b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4a73b1-7f9ccbd4c9-j9ws8"
Oct 31 21:55:31.481275 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:31.481216    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-4a73b1-77b884cb9-g55qb"]
Oct 31 21:55:31.493064 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:31.493029    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-flzxg\" (UniqueName: \"kubernetes.io/projected/e438fbbd-062a-4ba7-9c8e-fe954d0cf55b-kube-api-access-flzxg\") pod \"payload-pre-processing-e2e-extmod-b-4a73b1-7f9ccbd4c9-j9ws8\" (UID: \"e438fbbd-062a-4ba7-9c8e-fe954d0cf55b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4a73b1-7f9ccbd4c9-j9ws8"
Oct 31 21:55:31.534167 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:31.534123    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4a73b1-7f9ccbd4c9-j9ws8"
Oct 31 21:55:31.557820 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:31.557780    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7bl4\" (UniqueName: \"kubernetes.io/projected/48f52b01-f63f-47c2-b0a5-d07370934c67-kube-api-access-z7bl4\") pod \"payload-processing-e2e-extmod-b-4a73b1-77b884cb9-g55qb\" (UID: \"48f52b01-f63f-47c2-b0a5-d07370934c67\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-4a73b1-77b884cb9-g55qb"
Oct 31 21:55:31.558001 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:31.557981    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48f52b01-f63f-47c2-b0a5-d07370934c67-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-4a73b1-77b884cb9-g55qb\" (UID: \"48f52b01-f63f-47c2-b0a5-d07370934c67\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-4a73b1-77b884cb9-g55qb"
Oct 31 21:55:31.558687 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:31.558669    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48f52b01-f63f-47c2-b0a5-d07370934c67-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-4a73b1-77b884cb9-g55qb\" (UID: \"48f52b01-f63f-47c2-b0a5-d07370934c67\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-4a73b1-77b884cb9-g55qb"
Oct 31 21:55:31.583488 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:31.583454    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7bl4\" (UniqueName: \"kubernetes.io/projected/48f52b01-f63f-47c2-b0a5-d07370934c67-kube-api-access-z7bl4\") pod \"payload-processing-e2e-extmod-b-4a73b1-77b884cb9-g55qb\" (UID: \"48f52b01-f63f-47c2-b0a5-d07370934c67\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-4a73b1-77b884cb9-g55qb"
Oct 31 21:55:31.748997 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:31.748954    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-4a73b1-77b884cb9-g55qb"
Oct 31 21:55:31.842434 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:31.842395    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-4a73b1-7f9ccbd4c9-j9ws8"]
Oct 31 21:55:31.843787 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:55:31.843749    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode438fbbd_062a_4ba7_9c8e_fe954d0cf55b.slice/crio-5763aa2ad4b0d4d57c50a9cde3dfae654b97f04996a4da78d07ec648aad50b85 WatchSource:0}: Error finding container 5763aa2ad4b0d4d57c50a9cde3dfae654b97f04996a4da78d07ec648aad50b85: Status 404 returned error can't find the container with id 5763aa2ad4b0d4d57c50a9cde3dfae654b97f04996a4da78d07ec648aad50b85
Oct 31 21:55:32.058675 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:55:32.058647    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48f52b01_f63f_47c2_b0a5_d07370934c67.slice/crio-817c33831dc1e8105b36023fb07070800e635cd7c8b4a0380bf1e4d9975d327b WatchSource:0}: Error finding container 817c33831dc1e8105b36023fb07070800e635cd7c8b4a0380bf1e4d9975d327b: Status 404 returned error can't find the container with id 817c33831dc1e8105b36023fb07070800e635cd7c8b4a0380bf1e4d9975d327b
Oct 31 21:55:32.058935 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:32.058913    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-4a73b1-77b884cb9-g55qb"]
Oct 31 21:55:32.432841 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:32.432751    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:32.601416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:32.601377    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-4a73b1-77b884cb9-g55qb" event={"ID":"48f52b01-f63f-47c2-b0a5-d07370934c67","Type":"ContainerStarted","Data":"5c773c0198e8baf9f670550ca3cab563e260118751e84d5c89d52869889286c2"}
Oct 31 21:55:32.601650 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:32.601446    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-4a73b1-77b884cb9-g55qb" event={"ID":"48f52b01-f63f-47c2-b0a5-d07370934c67","Type":"ContainerStarted","Data":"817c33831dc1e8105b36023fb07070800e635cd7c8b4a0380bf1e4d9975d327b"}
Oct 31 21:55:32.601650 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:32.601475    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-4a73b1-77b884cb9-g55qb"
Oct 31 21:55:32.603321 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:32.603290    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4a73b1-7f9ccbd4c9-j9ws8" event={"ID":"e438fbbd-062a-4ba7-9c8e-fe954d0cf55b","Type":"ContainerStarted","Data":"f4a409df07b78cb97a4b37f78725d2cb5ad2e6fdb0ce2c7c8064c99c96842a81"}
Oct 31 21:55:32.603448 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:32.603336    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4a73b1-7f9ccbd4c9-j9ws8" event={"ID":"e438fbbd-062a-4ba7-9c8e-fe954d0cf55b","Type":"ContainerStarted","Data":"5763aa2ad4b0d4d57c50a9cde3dfae654b97f04996a4da78d07ec648aad50b85"}
Oct 31 21:55:32.603448 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:32.603387    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4a73b1-7f9ccbd4c9-j9ws8"
Oct 31 21:55:32.647013 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:32.646931    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-b-4a73b1-77b884cb9-g55qb" podStartSLOduration=1.64691871 podStartE2EDuration="1.64691871s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:55:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:55:32.644808648 +0000 UTC m=+2372.561118081" watchObservedRunningTime="2026-10-31 21:55:32.64691871 +0000 UTC m=+2372.563228140"
Oct 31 21:55:32.690305 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:32.690225    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4a73b1-7f9ccbd4c9-j9ws8" podStartSLOduration=1.690210102 podStartE2EDuration="1.690210102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:55:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:55:32.688518662 +0000 UTC m=+2372.604828093" watchObservedRunningTime="2026-10-31 21:55:32.690210102 +0000 UTC m=+2372.606519531"
Oct 31 21:55:34.120766 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:34.120734    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:35.913898 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:35.913867    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:37.664310 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:37.664271    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:39.374958 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:39.374903    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:39.982700 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:39.982666    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"]
Oct 31 21:55:39.987841 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:39.987817    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 21:55:40.029198 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.029167    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w0-5237cd-openshift-default-dockercfg-76npr\""
Oct 31 21:55:40.044292 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.034366    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istio-data\") pod \"e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") " pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 21:55:40.044292 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.034422    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-workload-certs\") pod \"e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") " pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 21:55:40.044292 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.034543    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istio-envoy\") pod \"e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") " pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 21:55:40.044292 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.034610    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-credential-socket\") pod \"e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") " pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 21:55:40.044292 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.034719    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-workload-socket\") pod \"e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") " pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 21:55:40.044292 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.034758    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istio-token\") pod \"e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") " pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 21:55:40.044292 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.034835    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istiod-ca-cert\") pod \"e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") " pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 21:55:40.044292 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.034883    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5qhh\" (UniqueName: \"kubernetes.io/projected/60161a74-1f84-4f09-9a66-b0c1d1fdce14-kube-api-access-q5qhh\") pod \"e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") " pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 21:55:40.044292 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.034988    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istio-podinfo\") pod \"e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") " pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 21:55:40.044292 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.037176    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"]
Oct 31 21:55:40.136379 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.136337    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istio-envoy\") pod \"e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") " pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 21:55:40.136548 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.136402    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-credential-socket\") pod \"e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") " pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 21:55:40.136548 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.136458    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-workload-socket\") pod \"e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") " pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 21:55:40.136548 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.136477    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istio-token\") pod \"e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") " pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 21:55:40.136548 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.136521    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istiod-ca-cert\") pod \"e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") " pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 21:55:40.136548 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.136545    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5qhh\" (UniqueName: \"kubernetes.io/projected/60161a74-1f84-4f09-9a66-b0c1d1fdce14-kube-api-access-q5qhh\") pod \"e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") " pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 21:55:40.136829 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.136594    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istio-podinfo\") pod \"e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") " pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 21:55:40.136829 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.136669    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istio-data\") pod \"e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") " pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 21:55:40.136829 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.136696    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-workload-certs\") pod \"e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") " pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 21:55:40.137037 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.136868    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-workload-socket\") pod \"e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") " pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 21:55:40.137037 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.136997    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-credential-socket\") pod \"e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") " pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 21:55:40.137273 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.137247    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istio-data\") pod \"e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") " pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 21:55:40.137367 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.137271    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istiod-ca-cert\") pod \"e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") " pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 21:55:40.137367 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.137316    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-workload-certs\") pod \"e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") " pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 21:55:40.138864 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.138829    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istio-envoy\") pod \"e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") " pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 21:55:40.139156 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.139138    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istio-podinfo\") pod \"e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") " pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 21:55:40.161456 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.161432    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istio-token\") pod \"e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") " pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 21:55:40.161791 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.161774    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5qhh\" (UniqueName: \"kubernetes.io/projected/60161a74-1f84-4f09-9a66-b0c1d1fdce14-kube-api-access-q5qhh\") pod \"e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") " pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 21:55:40.302921 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.302889    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 21:55:40.632139 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.632109    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"]
Oct 31 21:55:40.632470 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:55:40.632311    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60161a74_1f84_4f09_9a66_b0c1d1fdce14.slice/crio-be6b7dec75dacfb88de7f99fcdbaf5c8af1caa0f277078c0165b27af6b77c01b WatchSource:0}: Error finding container be6b7dec75dacfb88de7f99fcdbaf5c8af1caa0f277078c0165b27af6b77c01b: Status 404 returned error can't find the container with id be6b7dec75dacfb88de7f99fcdbaf5c8af1caa0f277078c0165b27af6b77c01b
Oct 31 21:55:40.635283 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.635249    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:55:40.635371 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.635316    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:55:40.635371 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.635352    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:55:40.663359 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:40.663331    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x" event={"ID":"60161a74-1f84-4f09-9a66-b0c1d1fdce14","Type":"ContainerStarted","Data":"be6b7dec75dacfb88de7f99fcdbaf5c8af1caa0f277078c0165b27af6b77c01b"}
Oct 31 21:55:41.134354 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:41.134294    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:41.672916 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:41.672875    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x" event={"ID":"60161a74-1f84-4f09-9a66-b0c1d1fdce14","Type":"ContainerStarted","Data":"03b7e02788741058f0b69f5b5a10c4ca570ec8f5a6bd65e4df21e02fe9b83f59"}
Oct 31 21:55:41.749863 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:41.749787    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x" podStartSLOduration=2.749768033 podStartE2EDuration="2.749768033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:55:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:55:41.748785894 +0000 UTC m=+2381.665095326" watchObservedRunningTime="2026-10-31 21:55:41.749768033 +0000 UTC m=+2381.666077467"
Oct 31 21:55:42.303993 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:42.303963    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 21:55:42.305837 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:42.305810    4460 patch_prober.go:29] interesting pod/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused" start-of-body=
Oct 31 21:55:42.305979 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:42.305859    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x" podUID="60161a74-1f84-4f09-9a66-b0c1d1fdce14" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused"
Oct 31 21:55:42.858860 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:42.858757    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:43.303590 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:43.303551    4460 patch_prober.go:29] interesting pod/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused" start-of-body=
Oct 31 21:55:43.303767 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:43.303615    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x" podUID="60161a74-1f84-4f09-9a66-b0c1d1fdce14" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused"
Oct 31 21:55:43.613520 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:43.613441    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4a73b1-7f9ccbd4c9-j9ws8"
Oct 31 21:55:43.613520 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:43.613517    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-4a73b1-77b884cb9-g55qb"
Oct 31 21:55:44.304018 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:44.303980    4460 patch_prober.go:29] interesting pod/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused" start-of-body=
Oct 31 21:55:44.304447 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:44.304039    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x" podUID="60161a74-1f84-4f09-9a66-b0c1d1fdce14" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused"
Oct 31 21:55:44.493994 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:44.493959    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:44.606351 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:44.606243    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:44.735967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:44.725362    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:45.303641 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:45.303602    4460 patch_prober.go:29] interesting pod/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused" start-of-body=
Oct 31 21:55:45.303824 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:45.303657    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x" podUID="60161a74-1f84-4f09-9a66-b0c1d1fdce14" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused"
Oct 31 21:55:45.620022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:45.619442    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:55:46.215372 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:46.215337    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:46.303952 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:46.303905    4460 patch_prober.go:29] interesting pod/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused" start-of-body=
Oct 31 21:55:46.304131 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:46.303970    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x" podUID="60161a74-1f84-4f09-9a66-b0c1d1fdce14" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.174:15021/healthz/ready\": dial tcp 10.133.0.174:15021: connect: connection refused"
Oct 31 21:55:47.389510 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:47.389429    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 21:55:47.390228 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:47.389783    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 21:55:47.391245 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:47.391205    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 21:55:47.986606 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:47.986567    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:48.435008 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:48.434962    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:49.709634 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:49.709595    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:51.415979 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:51.415924    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:51.491640 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:55:51.488630    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:55:53.127416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:53.127376    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:54.731345 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:54.731303    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:56.507377 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:56.507345    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:58.082817 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:58.082784    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:55:59.769625 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:55:59.769585    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:00.669538 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:00.669508    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ssl9_26069598-5dc4-4c96-b55f-dd2f09884419/ovn-acl-logging/0.log"
Oct 31 21:56:00.677510 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:00.677484    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ssl9_26069598-5dc4-4c96-b55f-dd2f09884419/ovn-acl-logging/0.log"
Oct 31 21:56:01.531842 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:01.531806    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:03.230074 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:03.230034    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:04.889445 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:04.889391    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:04.995168 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:04.995129    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:05.110112 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:05.110074    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:06.192164 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:06.192128    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:06.747528 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:06.747482    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:08.054855 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:08.054812    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:08.433579 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:08.433490    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:10.253736 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:10.253693    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:11.961953 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:11.961904    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:13.718601 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:13.718558    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:14.234320 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:14.234281    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:14.366318 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:14.366275    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:14.487111 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:14.487018    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:14.615021 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:14.612441    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:14.730434 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:14.730399    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:14.853760 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:14.853722    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:14.969964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:14.968346    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:15.085425 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:15.085397    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:15.489626 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:15.489587    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:15.635845 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:15.635798    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:15.817302 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:15.817263    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:16.153512 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:16.153397    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:16.289971 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:16.281901    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:17.166987 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:17.162541    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:17.273558 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:17.273523    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:17.709985 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:17.708626    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:17.821888 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:17.821840    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:17.958965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:17.955602    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:19.429531 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:19.429495    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq"]
Oct 31 21:56:19.435493 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:19.435473    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq"
Oct 31 21:56:19.452263 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:19.452230    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-56a6d4\"/\"agc-cred-ba36f972ebcb\""
Oct 31 21:56:19.455394 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:19.455361    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq"]
Oct 31 21:56:19.463530 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:19.463491    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-56a6d4\"/\"payload-processing-external-model-plugins-e2e-extmod-b-56a6d4\""
Oct 31 21:56:19.463695 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:19.463532    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-56a6d4\"/\"routing-overlay\""
Oct 31 21:56:19.463695 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:19.463643    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-56a6d4\"/\"kube-root-ca.crt\""
Oct 31 21:56:19.463695 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:19.463679    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-56a6d4\"/\"openshift-service-ca.crt\""
Oct 31 21:56:19.552731 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:19.552696    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-56a6d4\"/\"agc-cred-8162ebadb51a\""
Oct 31 21:56:19.552731 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:19.552712    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-56a6d4\"/\"payload-processing-external-model-e2e--34b55c0e-dockercfg-5bdlf\""
Oct 31 21:56:19.584798 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:19.584760    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/7c695450-20d4-49f3-b9f4-a522980c1f83-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq\" (UID: \"7c695450-20d4-49f3-b9f4-a522980c1f83\") " pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq"
Oct 31 21:56:19.585012 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:19.584969    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5chl\" (UniqueName: \"kubernetes.io/projected/7c695450-20d4-49f3-b9f4-a522980c1f83-kube-api-access-q5chl\") pod \"payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq\" (UID: \"7c695450-20d4-49f3-b9f4-a522980c1f83\") " pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq"
Oct 31 21:56:19.585012 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:19.585007    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7c695450-20d4-49f3-b9f4-a522980c1f83-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq\" (UID: \"7c695450-20d4-49f3-b9f4-a522980c1f83\") " pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq"
Oct 31 21:56:19.585139 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:19.585094    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c695450-20d4-49f3-b9f4-a522980c1f83-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq\" (UID: \"7c695450-20d4-49f3-b9f4-a522980c1f83\") " pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq"
Oct 31 21:56:19.686410 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:19.686322    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/7c695450-20d4-49f3-b9f4-a522980c1f83-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq\" (UID: \"7c695450-20d4-49f3-b9f4-a522980c1f83\") " pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq"
Oct 31 21:56:19.686585 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:19.686429    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5chl\" (UniqueName: \"kubernetes.io/projected/7c695450-20d4-49f3-b9f4-a522980c1f83-kube-api-access-q5chl\") pod \"payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq\" (UID: \"7c695450-20d4-49f3-b9f4-a522980c1f83\") " pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq"
Oct 31 21:56:19.686585 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:19.686454    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7c695450-20d4-49f3-b9f4-a522980c1f83-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq\" (UID: \"7c695450-20d4-49f3-b9f4-a522980c1f83\") " pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq"
Oct 31 21:56:19.686585 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:19.686486    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c695450-20d4-49f3-b9f4-a522980c1f83-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq\" (UID: \"7c695450-20d4-49f3-b9f4-a522980c1f83\") " pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq"
Oct 31 21:56:19.687138 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:19.687113    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7c695450-20d4-49f3-b9f4-a522980c1f83-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq\" (UID: \"7c695450-20d4-49f3-b9f4-a522980c1f83\") " pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq"
Oct 31 21:56:19.687292 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:19.687269    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c695450-20d4-49f3-b9f4-a522980c1f83-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq\" (UID: \"7c695450-20d4-49f3-b9f4-a522980c1f83\") " pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq"
Oct 31 21:56:19.688851 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:19.688830    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/7c695450-20d4-49f3-b9f4-a522980c1f83-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq\" (UID: \"7c695450-20d4-49f3-b9f4-a522980c1f83\") " pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq"
Oct 31 21:56:19.694966 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:19.694908    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:19.712846 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:19.712816    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5chl\" (UniqueName: \"kubernetes.io/projected/7c695450-20d4-49f3-b9f4-a522980c1f83-kube-api-access-q5chl\") pod \"payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq\" (UID: \"7c695450-20d4-49f3-b9f4-a522980c1f83\") " pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq"
Oct 31 21:56:19.751967 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:19.751277    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq"
Oct 31 21:56:20.087699 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:20.087662    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq"]
Oct 31 21:56:20.088405 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:56:20.088380    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c695450_20d4_49f3_b9f4_a522980c1f83.slice/crio-84faa7c58ca1c8fc32d810117754c1966ffb7750756815969d9269a89efd7b78 WatchSource:0}: Error finding container 84faa7c58ca1c8fc32d810117754c1966ffb7750756815969d9269a89efd7b78: Status 404 returned error can't find the container with id 84faa7c58ca1c8fc32d810117754c1966ffb7750756815969d9269a89efd7b78
Oct 31 21:56:21.095626 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:21.095586    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq" event={"ID":"7c695450-20d4-49f3-b9f4-a522980c1f83","Type":"ContainerStarted","Data":"03a8a4ca52be07e1ea5ad2165f123b4abeb8891dca61f182e5e3c23e508323fb"}
Oct 31 21:56:21.096067 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:21.095687    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq"
Oct 31 21:56:21.096067 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:21.095707    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq" event={"ID":"7c695450-20d4-49f3-b9f4-a522980c1f83","Type":"ContainerStarted","Data":"84faa7c58ca1c8fc32d810117754c1966ffb7750756815969d9269a89efd7b78"}
Oct 31 21:56:21.141530 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:21.141473    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq" podStartSLOduration=2.141457633 podStartE2EDuration="2.141457633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:56:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:56:21.140427182 +0000 UTC m=+2421.056736611" watchObservedRunningTime="2026-10-31 21:56:21.141457633 +0000 UTC m=+2421.057767063"
Oct 31 21:56:21.431797 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:21.431697    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:21.455015 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:56:21.454978    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:56:23.102699 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:23.102662    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:24.740375 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:24.740333    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:26.510107 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:26.510066    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:26.623288 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:26.623250    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:27.830934 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:27.830892    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:28.140727 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:28.140617    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:28.269858 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:28.269813    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:28.381078 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:28.381030    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:29.845784 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:29.845747    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:31.314192 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:31.314151    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5"]
Oct 31 21:56:31.728566 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:31.728470    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:31.865007 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:31.864932    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:32.106484 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:32.106447    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq"
Oct 31 21:56:33.440789 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:33.440749    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:33.562631 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:33.562590    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:33.612823 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:33.612790    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-2dbae9-6ccd966867-7qlsd"]
Oct 31 21:56:33.617934 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:33.617907    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-2dbae9-6ccd966867-7qlsd"
Oct 31 21:56:33.637346 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:33.637312    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-2dbae9-6ccd966867-7qlsd"]
Oct 31 21:56:33.645997 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:33.645923    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-2dbae9\""
Oct 31 21:56:33.646269 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:33.646251    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-2dbae9-dockercfg-g5rdn\""
Oct 31 21:56:33.671363 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:33.671330    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:33.750024 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:33.743142    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb66365f-35ce-4bb0-a756-9c2fba850b58-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-2dbae9-6ccd966867-7qlsd\" (UID: \"eb66365f-35ce-4bb0-a756-9c2fba850b58\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-2dbae9-6ccd966867-7qlsd"
Oct 31 21:56:33.773963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:33.769376    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fn5ct\" (UniqueName: \"kubernetes.io/projected/eb66365f-35ce-4bb0-a756-9c2fba850b58-kube-api-access-fn5ct\") pod \"payload-pre-processing-e2e-extmod-b-2dbae9-6ccd966867-7qlsd\" (UID: \"eb66365f-35ce-4bb0-a756-9c2fba850b58\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-2dbae9-6ccd966867-7qlsd"
Oct 31 21:56:33.820046 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:33.820006    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-2dbae9-7dd5d8cd79-wjpd6"]
Oct 31 21:56:33.842974 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:33.826189    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-2dbae9-7dd5d8cd79-wjpd6"
Oct 31 21:56:33.892173 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:33.889194    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb66365f-35ce-4bb0-a756-9c2fba850b58-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-2dbae9-6ccd966867-7qlsd\" (UID: \"eb66365f-35ce-4bb0-a756-9c2fba850b58\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-2dbae9-6ccd966867-7qlsd"
Oct 31 21:56:33.892173 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:33.889306    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fn5ct\" (UniqueName: \"kubernetes.io/projected/eb66365f-35ce-4bb0-a756-9c2fba850b58-kube-api-access-fn5ct\") pod \"payload-pre-processing-e2e-extmod-b-2dbae9-6ccd966867-7qlsd\" (UID: \"eb66365f-35ce-4bb0-a756-9c2fba850b58\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-2dbae9-6ccd966867-7qlsd"
Oct 31 21:56:33.892173 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:33.890842    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb66365f-35ce-4bb0-a756-9c2fba850b58-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-2dbae9-6ccd966867-7qlsd\" (UID: \"eb66365f-35ce-4bb0-a756-9c2fba850b58\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-2dbae9-6ccd966867-7qlsd"
Oct 31 21:56:33.911898 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:33.911861    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-2dbae9-7dd5d8cd79-wjpd6"]
Oct 31 21:56:33.915526 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:33.915494    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn5ct\" (UniqueName: \"kubernetes.io/projected/eb66365f-35ce-4bb0-a756-9c2fba850b58-kube-api-access-fn5ct\") pod \"payload-pre-processing-e2e-extmod-b-2dbae9-6ccd966867-7qlsd\" (UID: \"eb66365f-35ce-4bb0-a756-9c2fba850b58\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-2dbae9-6ccd966867-7qlsd"
Oct 31 21:56:33.936280 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:33.936249    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-2dbae9-6ccd966867-7qlsd"
Oct 31 21:56:33.991731 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:33.991546    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b79f9530-e471-49ab-acce-8391f7b8a91a-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-2dbae9-7dd5d8cd79-wjpd6\" (UID: \"b79f9530-e471-49ab-acce-8391f7b8a91a\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-2dbae9-7dd5d8cd79-wjpd6"
Oct 31 21:56:33.991731 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:33.991703    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92jq4\" (UniqueName: \"kubernetes.io/projected/b79f9530-e471-49ab-acce-8391f7b8a91a-kube-api-access-92jq4\") pod \"payload-processing-e2e-extmod-b-2dbae9-7dd5d8cd79-wjpd6\" (UID: \"b79f9530-e471-49ab-acce-8391f7b8a91a\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-2dbae9-7dd5d8cd79-wjpd6"
Oct 31 21:56:34.092781 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:34.092740    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92jq4\" (UniqueName: \"kubernetes.io/projected/b79f9530-e471-49ab-acce-8391f7b8a91a-kube-api-access-92jq4\") pod \"payload-processing-e2e-extmod-b-2dbae9-7dd5d8cd79-wjpd6\" (UID: \"b79f9530-e471-49ab-acce-8391f7b8a91a\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-2dbae9-7dd5d8cd79-wjpd6"
Oct 31 21:56:34.092991 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:34.092971    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b79f9530-e471-49ab-acce-8391f7b8a91a-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-2dbae9-7dd5d8cd79-wjpd6\" (UID: \"b79f9530-e471-49ab-acce-8391f7b8a91a\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-2dbae9-7dd5d8cd79-wjpd6"
Oct 31 21:56:34.093721 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:34.093699    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b79f9530-e471-49ab-acce-8391f7b8a91a-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-2dbae9-7dd5d8cd79-wjpd6\" (UID: \"b79f9530-e471-49ab-acce-8391f7b8a91a\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-2dbae9-7dd5d8cd79-wjpd6"
Oct 31 21:56:34.118100 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:34.118066    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-92jq4\" (UniqueName: \"kubernetes.io/projected/b79f9530-e471-49ab-acce-8391f7b8a91a-kube-api-access-92jq4\") pod \"payload-processing-e2e-extmod-b-2dbae9-7dd5d8cd79-wjpd6\" (UID: \"b79f9530-e471-49ab-acce-8391f7b8a91a\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-2dbae9-7dd5d8cd79-wjpd6"
Oct 31 21:56:34.254366 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:34.254335    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-2dbae9-7dd5d8cd79-wjpd6"
Oct 31 21:56:34.272458 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:34.272427    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-2dbae9-6ccd966867-7qlsd"]
Oct 31 21:56:34.273262 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:56:34.273236    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb66365f_35ce_4bb0_a756_9c2fba850b58.slice/crio-310ff336c351e29a52c458793bc1fb167fb5c3c6b25fa162193b486d59afaed4 WatchSource:0}: Error finding container 310ff336c351e29a52c458793bc1fb167fb5c3c6b25fa162193b486d59afaed4: Status 404 returned error can't find the container with id 310ff336c351e29a52c458793bc1fb167fb5c3c6b25fa162193b486d59afaed4
Oct 31 21:56:34.289843 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:34.288175    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:34.584383 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:34.584354    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-2dbae9-7dd5d8cd79-wjpd6"]
Oct 31 21:56:34.585062 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:56:34.585034    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb79f9530_e471_49ab_acce_8391f7b8a91a.slice/crio-c0ac3f36533f48df6a914e1b76463f6f422dbd2fed2786a44d8a593ee377b5e6 WatchSource:0}: Error finding container c0ac3f36533f48df6a914e1b76463f6f422dbd2fed2786a44d8a593ee377b5e6: Status 404 returned error can't find the container with id c0ac3f36533f48df6a914e1b76463f6f422dbd2fed2786a44d8a593ee377b5e6
Oct 31 21:56:35.203303 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:35.203269    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-2dbae9-6ccd966867-7qlsd" event={"ID":"eb66365f-35ce-4bb0-a756-9c2fba850b58","Type":"ContainerStarted","Data":"b09830e046b470f4de06274d04307fc9418f44f4c1493f491cff2460247835a9"}
Oct 31 21:56:35.203522 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:35.203327    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-2dbae9-6ccd966867-7qlsd" event={"ID":"eb66365f-35ce-4bb0-a756-9c2fba850b58","Type":"ContainerStarted","Data":"310ff336c351e29a52c458793bc1fb167fb5c3c6b25fa162193b486d59afaed4"}
Oct 31 21:56:35.203522 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:35.203357    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-2dbae9-6ccd966867-7qlsd"
Oct 31 21:56:35.205028 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:35.204998    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-2dbae9-7dd5d8cd79-wjpd6" event={"ID":"b79f9530-e471-49ab-acce-8391f7b8a91a","Type":"ContainerStarted","Data":"e465aabd545c7154841b88aec6fb552a5ba3a6eef65a95bda6cab7ef84c9bd2e"}
Oct 31 21:56:35.205152 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:35.205052    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-2dbae9-7dd5d8cd79-wjpd6"
Oct 31 21:56:35.205152 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:35.205073    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-2dbae9-7dd5d8cd79-wjpd6" event={"ID":"b79f9530-e471-49ab-acce-8391f7b8a91a","Type":"ContainerStarted","Data":"c0ac3f36533f48df6a914e1b76463f6f422dbd2fed2786a44d8a593ee377b5e6"}
Oct 31 21:56:35.252004 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:35.251923    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-2dbae9-6ccd966867-7qlsd" podStartSLOduration=2.251908672 podStartE2EDuration="2.251908672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:56:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:56:35.247706509 +0000 UTC m=+2435.164015938" watchObservedRunningTime="2026-10-31 21:56:35.251908672 +0000 UTC m=+2435.168218101"
Oct 31 21:56:35.294536 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:35.294467    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-b-2dbae9-7dd5d8cd79-wjpd6" podStartSLOduration=2.294448651 podStartE2EDuration="2.294448651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:56:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:56:35.291056786 +0000 UTC m=+2435.207366216" watchObservedRunningTime="2026-10-31 21:56:35.294448651 +0000 UTC m=+2435.210758080"
Oct 31 21:56:35.505173 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:35.505075    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:36.314781 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:36.314739    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5" podUID="2b649c1b-7735-4edf-8ec6-c04cc500d5c2" containerName="payload-processing" containerID="cri-o://0d1fb174a012a9f6e3eba101af4fff7f301a009a13ece4b7762d73d9a0080184" gracePeriod=25
Oct 31 21:56:36.599954 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:36.599914    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5"
Oct 31 21:56:36.641690 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:36.641656    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2b649c1b-7735-4edf-8ec6-c04cc500d5c2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b649c1b-7735-4edf-8ec6-c04cc500d5c2-plugins-config-volume\") pod \"2b649c1b-7735-4edf-8ec6-c04cc500d5c2\" (UID: \"2b649c1b-7735-4edf-8ec6-c04cc500d5c2\") "
Oct 31 21:56:36.641871 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:36.641798    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b649c1b-7735-4edf-8ec6-c04cc500d5c2-provider-credentials\" (UniqueName: \"kubernetes.io/projected/2b649c1b-7735-4edf-8ec6-c04cc500d5c2-provider-credentials\") pod \"2b649c1b-7735-4edf-8ec6-c04cc500d5c2\" (UID: \"2b649c1b-7735-4edf-8ec6-c04cc500d5c2\") "
Oct 31 21:56:36.642018 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:36.641999    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b649c1b-7735-4edf-8ec6-c04cc500d5c2-kube-api-access-ntvtj\" (UniqueName: \"kubernetes.io/projected/2b649c1b-7735-4edf-8ec6-c04cc500d5c2-kube-api-access-ntvtj\") pod \"2b649c1b-7735-4edf-8ec6-c04cc500d5c2\" (UID: \"2b649c1b-7735-4edf-8ec6-c04cc500d5c2\") "
Oct 31 21:56:36.642098 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:36.642053    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2b649c1b-7735-4edf-8ec6-c04cc500d5c2-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2b649c1b-7735-4edf-8ec6-c04cc500d5c2-routing-overlay-volume\") pod \"2b649c1b-7735-4edf-8ec6-c04cc500d5c2\" (UID: \"2b649c1b-7735-4edf-8ec6-c04cc500d5c2\") "
Oct 31 21:56:36.642098 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:36.642062    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b649c1b-7735-4edf-8ec6-c04cc500d5c2-plugins-config-volume" pod "2b649c1b-7735-4edf-8ec6-c04cc500d5c2" (UID: "2b649c1b-7735-4edf-8ec6-c04cc500d5c2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:56:36.642617 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:36.642587    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b649c1b-7735-4edf-8ec6-c04cc500d5c2-routing-overlay-volume" pod "2b649c1b-7735-4edf-8ec6-c04cc500d5c2" (UID: "2b649c1b-7735-4edf-8ec6-c04cc500d5c2"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:56:36.643242 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:36.643218    4460 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2b649c1b-7735-4edf-8ec6-c04cc500d5c2-routing-overlay-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:56:36.643353 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:36.643245    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b649c1b-7735-4edf-8ec6-c04cc500d5c2-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:56:36.647610 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:36.646345    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b649c1b-7735-4edf-8ec6-c04cc500d5c2-provider-credentials" pod "2b649c1b-7735-4edf-8ec6-c04cc500d5c2" (UID: "2b649c1b-7735-4edf-8ec6-c04cc500d5c2"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:56:36.649447 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:36.649405    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b649c1b-7735-4edf-8ec6-c04cc500d5c2-kube-api-access-ntvtj" pod "2b649c1b-7735-4edf-8ec6-c04cc500d5c2" (UID: "2b649c1b-7735-4edf-8ec6-c04cc500d5c2"). InnerVolumeSpecName "kube-api-access-ntvtj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:56:36.744536 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:36.744497    4460 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/2b649c1b-7735-4edf-8ec6-c04cc500d5c2-provider-credentials\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:56:36.744536 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:36.744530    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ntvtj\" (UniqueName: \"kubernetes.io/projected/2b649c1b-7735-4edf-8ec6-c04cc500d5c2-kube-api-access-ntvtj\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:56:37.221134 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:37.221095    4460 generic.go:356] "Generic (PLEG): container finished" podID="2b649c1b-7735-4edf-8ec6-c04cc500d5c2" containerID="0d1fb174a012a9f6e3eba101af4fff7f301a009a13ece4b7762d73d9a0080184" exitCode=0
Oct 31 21:56:37.221134 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:37.221129    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5"
Oct 31 21:56:37.221418 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:37.221172    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5" event={"ID":"2b649c1b-7735-4edf-8ec6-c04cc500d5c2","Type":"ContainerDied","Data":"0d1fb174a012a9f6e3eba101af4fff7f301a009a13ece4b7762d73d9a0080184"}
Oct 31 21:56:37.221418 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:37.221229    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5" event={"ID":"2b649c1b-7735-4edf-8ec6-c04cc500d5c2","Type":"ContainerDied","Data":"6cb9c87357a22e3de93b4785da1b1dbc6eeb731ad6289ec10fbaa346adf33138"}
Oct 31 21:56:37.221418 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:37.221259    4460 scope.go:122] "RemoveContainer" containerID="0d1fb174a012a9f6e3eba101af4fff7f301a009a13ece4b7762d73d9a0080184"
Oct 31 21:56:37.234987 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:37.234965    4460 scope.go:122] "RemoveContainer" containerID="0d1fb174a012a9f6e3eba101af4fff7f301a009a13ece4b7762d73d9a0080184"
Oct 31 21:56:37.235311 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:56:37.235283    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d1fb174a012a9f6e3eba101af4fff7f301a009a13ece4b7762d73d9a0080184\": container with ID starting with 0d1fb174a012a9f6e3eba101af4fff7f301a009a13ece4b7762d73d9a0080184 not found: ID does not exist" containerID="0d1fb174a012a9f6e3eba101af4fff7f301a009a13ece4b7762d73d9a0080184"
Oct 31 21:56:37.235375 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:37.235322    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d1fb174a012a9f6e3eba101af4fff7f301a009a13ece4b7762d73d9a0080184"} err="failed to get container status \"0d1fb174a012a9f6e3eba101af4fff7f301a009a13ece4b7762d73d9a0080184\": rpc error: code = NotFound desc = could not find container \"0d1fb174a012a9f6e3eba101af4fff7f301a009a13ece4b7762d73d9a0080184\": container with ID starting with 0d1fb174a012a9f6e3eba101af4fff7f301a009a13ece4b7762d73d9a0080184 not found: ID does not exist"
Oct 31 21:56:37.240621 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:37.240596    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:37.335260 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:37.335220    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5"]
Oct 31 21:56:37.344164 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:37.344126    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45x5tn5"]
Oct 31 21:56:37.351676 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:56:37.351636    4460 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b649c1b_7735_4edf_8ec6_c04cc500d5c2.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b649c1b_7735_4edf_8ec6_c04cc500d5c2.slice/crio-6cb9c87357a22e3de93b4785da1b1dbc6eeb731ad6289ec10fbaa346adf33138\": RecentStats: unable to find data in memory cache]"
Oct 31 21:56:38.226227 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:38.226193    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2b649c1b-7735-4edf-8ec6-c04cc500d5c2" path="/var/lib/kubelet/pods/2b649c1b-7735-4edf-8ec6-c04cc500d5c2/volumes"
Oct 31 21:56:38.927180 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:38.927136    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:40.641191 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:40.641154    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:41.792279 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:41.792241    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:56:42.314323 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:42.314284    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:44.208525 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:44.208485    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:45.111550 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:45.111511    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:45.222336 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:45.222305    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:45.341709 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:45.341582    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:46.001369 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:46.001332    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:46.214910 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:46.214878    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-2dbae9-6ccd966867-7qlsd"
Oct 31 21:56:46.215392 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:46.215372    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-2dbae9-7dd5d8cd79-wjpd6"
Oct 31 21:56:46.771040 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:46.770976    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:47.123317 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:47.123237    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm"]
Oct 31 21:56:47.124109 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:47.124091    4460 cpu_manager.go:414] "containerMap: removing container" podUID="2b649c1b-7735-4edf-8ec6-c04cc500d5c2" containerName="payload-processing"
Oct 31 21:56:47.124168 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:47.124110    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b649c1b-7735-4edf-8ec6-c04cc500d5c2" containerName="payload-processing"
Oct 31 21:56:47.124253 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:47.124242    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b649c1b-7735-4edf-8ec6-c04cc500d5c2" containerName="payload-processing"
Oct 31 21:56:47.129578 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:47.129556    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm"
Oct 31 21:56:47.147247 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:47.147216    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm"]
Oct 31 21:56:47.156660 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:47.156632    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-c34cbd\"/\"agc-cred-0185341c205f\""
Oct 31 21:56:47.156796 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:47.156770    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-c34cbd\"/\"agc-cred-3d39da95e8af\""
Oct 31 21:56:47.156842 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:47.156832    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-c34cbd\"/\"agc-cred-145bf47370f9\""
Oct 31 21:56:47.157148 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:47.157118    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-c34cbd\"/\"payload-processing-external-model-e2e--bb6f7797-dockercfg-6p5zs\""
Oct 31 21:56:47.157691 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:47.157670    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-c34cbd\"/\"payload-processing-external-model-plugins-e2e-extmod-b-c34cbd\""
Oct 31 21:56:47.159891 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:47.159867    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-c34cbd\"/\"routing-overlay\""
Oct 31 21:56:47.160081 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:47.160061    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-c34cbd\"/\"openshift-service-ca.crt\""
Oct 31 21:56:47.160154 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:47.160109    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-c34cbd\"/\"kube-root-ca.crt\""
Oct 31 21:56:47.185243 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:47.185213    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kprhv\" (UniqueName: \"kubernetes.io/projected/54c25d6d-76c7-422d-b0a4-9bc3281dc818-kube-api-access-kprhv\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm\" (UID: \"54c25d6d-76c7-422d-b0a4-9bc3281dc818\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm"
Oct 31 21:56:47.185382 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:47.185361    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/54c25d6d-76c7-422d-b0a4-9bc3281dc818-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm\" (UID: \"54c25d6d-76c7-422d-b0a4-9bc3281dc818\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm"
Oct 31 21:56:47.185538 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:47.185513    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54c25d6d-76c7-422d-b0a4-9bc3281dc818-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm\" (UID: \"54c25d6d-76c7-422d-b0a4-9bc3281dc818\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm"
Oct 31 21:56:47.185612 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:47.185594    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/54c25d6d-76c7-422d-b0a4-9bc3281dc818-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm\" (UID: \"54c25d6d-76c7-422d-b0a4-9bc3281dc818\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm"
Oct 31 21:56:47.287288 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:47.287252    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54c25d6d-76c7-422d-b0a4-9bc3281dc818-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm\" (UID: \"54c25d6d-76c7-422d-b0a4-9bc3281dc818\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm"
Oct 31 21:56:47.287487 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:47.287315    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/54c25d6d-76c7-422d-b0a4-9bc3281dc818-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm\" (UID: \"54c25d6d-76c7-422d-b0a4-9bc3281dc818\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm"
Oct 31 21:56:47.287487 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:47.287441    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kprhv\" (UniqueName: \"kubernetes.io/projected/54c25d6d-76c7-422d-b0a4-9bc3281dc818-kube-api-access-kprhv\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm\" (UID: \"54c25d6d-76c7-422d-b0a4-9bc3281dc818\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm"
Oct 31 21:56:47.287582 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:47.287557    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/54c25d6d-76c7-422d-b0a4-9bc3281dc818-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm\" (UID: \"54c25d6d-76c7-422d-b0a4-9bc3281dc818\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm"
Oct 31 21:56:47.288088 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:47.288062    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54c25d6d-76c7-422d-b0a4-9bc3281dc818-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm\" (UID: \"54c25d6d-76c7-422d-b0a4-9bc3281dc818\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm"
Oct 31 21:56:47.288184 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:47.288165    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/54c25d6d-76c7-422d-b0a4-9bc3281dc818-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm\" (UID: \"54c25d6d-76c7-422d-b0a4-9bc3281dc818\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm"
Oct 31 21:56:47.290134 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:47.290113    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/54c25d6d-76c7-422d-b0a4-9bc3281dc818-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm\" (UID: \"54c25d6d-76c7-422d-b0a4-9bc3281dc818\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm"
Oct 31 21:56:47.312548 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:47.312517    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kprhv\" (UniqueName: \"kubernetes.io/projected/54c25d6d-76c7-422d-b0a4-9bc3281dc818-kube-api-access-kprhv\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm\" (UID: \"54c25d6d-76c7-422d-b0a4-9bc3281dc818\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm"
Oct 31 21:56:47.445225 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:47.445134    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm"
Oct 31 21:56:47.755416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:47.755382    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm"]
Oct 31 21:56:47.755670 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:56:47.755639    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54c25d6d_76c7_422d_b0a4_9bc3281dc818.slice/crio-c29028491493632993bebc2dd6ea376659bba1d9340fd23a587c332df03ea1dd WatchSource:0}: Error finding container c29028491493632993bebc2dd6ea376659bba1d9340fd23a587c332df03ea1dd: Status 404 returned error can't find the container with id c29028491493632993bebc2dd6ea376659bba1d9340fd23a587c332df03ea1dd
Oct 31 21:56:47.925034 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:47.925003    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:48.331509 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:48.331472    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm" event={"ID":"54c25d6d-76c7-422d-b0a4-9bc3281dc818","Type":"ContainerStarted","Data":"4b2e085faaba9164524e2a656f5d105a05f620aa6192366b4bd94db072df4978"}
Oct 31 21:56:48.331707 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:48.331541    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm"
Oct 31 21:56:48.331707 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:48.331563    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm" event={"ID":"54c25d6d-76c7-422d-b0a4-9bc3281dc818","Type":"ContainerStarted","Data":"c29028491493632993bebc2dd6ea376659bba1d9340fd23a587c332df03ea1dd"}
Oct 31 21:56:48.378099 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:48.378046    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm" podStartSLOduration=1.378029958 podStartE2EDuration="1.378029958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:56:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:56:48.375581109 +0000 UTC m=+2448.291890538" watchObservedRunningTime="2026-10-31 21:56:48.378029958 +0000 UTC m=+2448.294339387"
Oct 31 21:56:48.557466 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:48.557432    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:49.590799 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:49.590763    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:51.000354 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:51.000315    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:51.240219 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:51.240182    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:51.453451 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:56:51.453409    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:56:52.157325 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:52.157281    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:52.275223 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:52.275191    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:52.438470 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:52.438383    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:52.556962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:52.556804    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:52.754857 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:52.754772    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:52.918043 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:52.918011    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:53.033499 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:53.033459    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:53.159513 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:53.159477    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:53.525089 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:53.525051    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:53.638419 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:53.638369    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:53.772582 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:53.772544    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"]
Oct 31 21:56:53.772873 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:53.772823    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4" podUID="6870d753-72cd-409c-947b-442703ac27d0" containerName="istio-proxy" containerID="cri-o://23fd0c583d448e7d048f473c59bd2615b895e141a3d476292873e3703cda20fd" gracePeriod=30
Oct 31 21:56:54.260726 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:54.260688    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:54.951229 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:54.951182    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:55.078619 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:55.078565    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:56.628099 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:56.628068    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:58.305734 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:58.305698    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:56:59.057132 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.057063    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:56:59.175191 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.175149    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-credential-socket\") pod \"6870d753-72cd-409c-947b-442703ac27d0\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") "
Oct 31 21:56:59.175191 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.175193    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-istio-data\") pod \"6870d753-72cd-409c-947b-442703ac27d0\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") "
Oct 31 21:56:59.175465 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.175229    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6870d753-72cd-409c-947b-442703ac27d0-istio-token\" (UniqueName: \"kubernetes.io/projected/6870d753-72cd-409c-947b-442703ac27d0-istio-token\") pod \"6870d753-72cd-409c-947b-442703ac27d0\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") "
Oct 31 21:56:59.175465 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.175310    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6870d753-72cd-409c-947b-442703ac27d0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6870d753-72cd-409c-947b-442703ac27d0-istiod-ca-cert\") pod \"6870d753-72cd-409c-947b-442703ac27d0\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") "
Oct 31 21:56:59.175465 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.175330    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6870d753-72cd-409c-947b-442703ac27d0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6870d753-72cd-409c-947b-442703ac27d0-istio-podinfo\") pod \"6870d753-72cd-409c-947b-442703ac27d0\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") "
Oct 31 21:56:59.175465 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.175351    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-workload-socket\") pod \"6870d753-72cd-409c-947b-442703ac27d0\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") "
Oct 31 21:56:59.175465 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.175452    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6870d753-72cd-409c-947b-442703ac27d0-kube-api-access-cqlzn\" (UniqueName: \"kubernetes.io/projected/6870d753-72cd-409c-947b-442703ac27d0-kube-api-access-cqlzn\") pod \"6870d753-72cd-409c-947b-442703ac27d0\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") "
Oct 31 21:56:59.175743 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.175491    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-credential-socket" pod "6870d753-72cd-409c-947b-442703ac27d0" (UID: "6870d753-72cd-409c-947b-442703ac27d0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:56:59.175743 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.175533    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-workload-certs\") pod \"6870d753-72cd-409c-947b-442703ac27d0\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") "
Oct 31 21:56:59.175743 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.175537    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-istio-data" pod "6870d753-72cd-409c-947b-442703ac27d0" (UID: "6870d753-72cd-409c-947b-442703ac27d0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:56:59.175743 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.175616    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-istio-envoy\") pod \"6870d753-72cd-409c-947b-442703ac27d0\" (UID: \"6870d753-72cd-409c-947b-442703ac27d0\") "
Oct 31 21:56:59.175743 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.175729    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-workload-socket" pod "6870d753-72cd-409c-947b-442703ac27d0" (UID: "6870d753-72cd-409c-947b-442703ac27d0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:56:59.176056 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.175907    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-workload-certs" pod "6870d753-72cd-409c-947b-442703ac27d0" (UID: "6870d753-72cd-409c-947b-442703ac27d0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:56:59.176056 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.175839    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6870d753-72cd-409c-947b-442703ac27d0-istiod-ca-cert" pod "6870d753-72cd-409c-947b-442703ac27d0" (UID: "6870d753-72cd-409c-947b-442703ac27d0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:56:59.176342 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.176318    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6870d753-72cd-409c-947b-442703ac27d0-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:56:59.176421 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.176348    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:56:59.176421 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.176363    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:56:59.176421 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.176378    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:56:59.176421 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.176390    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:56:59.177838 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.177812    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-istio-envoy" pod "6870d753-72cd-409c-947b-442703ac27d0" (UID: "6870d753-72cd-409c-947b-442703ac27d0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:56:59.177965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.177836    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6870d753-72cd-409c-947b-442703ac27d0-istio-token" pod "6870d753-72cd-409c-947b-442703ac27d0" (UID: "6870d753-72cd-409c-947b-442703ac27d0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:56:59.177965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.177834    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6870d753-72cd-409c-947b-442703ac27d0-istio-podinfo" pod "6870d753-72cd-409c-947b-442703ac27d0" (UID: "6870d753-72cd-409c-947b-442703ac27d0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:56:59.177965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.177821    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6870d753-72cd-409c-947b-442703ac27d0-kube-api-access-cqlzn" pod "6870d753-72cd-409c-947b-442703ac27d0" (UID: "6870d753-72cd-409c-947b-442703ac27d0"). InnerVolumeSpecName "kube-api-access-cqlzn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:56:59.278370 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.278328    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6870d753-72cd-409c-947b-442703ac27d0-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:56:59.278370 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.278361    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6870d753-72cd-409c-947b-442703ac27d0-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:56:59.278370 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.278371    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqlzn\" (UniqueName: \"kubernetes.io/projected/6870d753-72cd-409c-947b-442703ac27d0-kube-api-access-cqlzn\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:56:59.278370 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.278381    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6870d753-72cd-409c-947b-442703ac27d0-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:56:59.342198 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.342102    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm"
Oct 31 21:56:59.441493 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.441452    4460 generic.go:356] "Generic (PLEG): container finished" podID="6870d753-72cd-409c-947b-442703ac27d0" containerID="23fd0c583d448e7d048f473c59bd2615b895e141a3d476292873e3703cda20fd" exitCode=0
Oct 31 21:56:59.441696 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.441510    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"
Oct 31 21:56:59.441696 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.441545    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4" event={"ID":"6870d753-72cd-409c-947b-442703ac27d0","Type":"ContainerDied","Data":"23fd0c583d448e7d048f473c59bd2615b895e141a3d476292873e3703cda20fd"}
Oct 31 21:56:59.441696 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.441629    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4" event={"ID":"6870d753-72cd-409c-947b-442703ac27d0","Type":"ContainerDied","Data":"b144f2cd36ae669b150c807ff5a50aba4aba66804545f71d8b6e2bb6ba85ed0c"}
Oct 31 21:56:59.441696 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.441681    4460 scope.go:122] "RemoveContainer" containerID="23fd0c583d448e7d048f473c59bd2615b895e141a3d476292873e3703cda20fd"
Oct 31 21:56:59.458723 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.458698    4460 scope.go:122] "RemoveContainer" containerID="23fd0c583d448e7d048f473c59bd2615b895e141a3d476292873e3703cda20fd"
Oct 31 21:56:59.459576 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:56:59.459551    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23fd0c583d448e7d048f473c59bd2615b895e141a3d476292873e3703cda20fd\": container with ID starting with 23fd0c583d448e7d048f473c59bd2615b895e141a3d476292873e3703cda20fd not found: ID does not exist" containerID="23fd0c583d448e7d048f473c59bd2615b895e141a3d476292873e3703cda20fd"
Oct 31 21:56:59.459672 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.459587    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23fd0c583d448e7d048f473c59bd2615b895e141a3d476292873e3703cda20fd"} err="failed to get container status \"23fd0c583d448e7d048f473c59bd2615b895e141a3d476292873e3703cda20fd\": rpc error: code = NotFound desc = could not find container \"23fd0c583d448e7d048f473c59bd2615b895e141a3d476292873e3703cda20fd\": container with ID starting with 23fd0c583d448e7d048f473c59bd2615b895e141a3d476292873e3703cda20fd not found: ID does not exist"
Oct 31 21:56:59.501314 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.501290    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"]
Oct 31 21:56:59.504973 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:56:59.504934    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mkgb4"]
Oct 31 21:57:00.034387 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:00.034329    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:00.229505 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:00.229465    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6870d753-72cd-409c-947b-442703ac27d0" path="/var/lib/kubelet/pods/6870d753-72cd-409c-947b-442703ac27d0/volumes"
Oct 31 21:57:00.383824 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:00.383745    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:00.493532 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:00.493500    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:00.599425 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:00.599390    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:01.627696 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:01.627657    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:03.236575 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:03.236538    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:04.936031 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:04.935987    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:05.675416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:05.675379    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:05.784053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:05.784016    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:06.736685 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:06.736649    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:07.314291 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:07.314247    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:08.424956 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:08.424910    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:10.095167 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:10.095124    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:10.208310 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:10.208268    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:10.318648 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:10.318611    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:12.011422 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:12.011381    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:13.821266 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:13.821178    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:13.952688 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:13.952654    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:14.072841 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:14.072749    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:14.182090 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:14.182051    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:14.418448 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:14.418351    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:15.049539 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:15.049501    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:15.157360 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:15.157317    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:15.709968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:15.709921    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:16.295386 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:16.295337    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:16.424546 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:16.424492    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:17.405495 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:17.405458    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:18.975522 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:18.975474    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:19.561012 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:19.560960    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:19.673599 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:19.673563    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:19.797876 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:19.797836    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:20.222537 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:20.222497    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:21.454080 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:57:21.454044    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:57:21.670830 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:21.670790    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:23.448694 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:23.448656    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:25.265724 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:25.265683    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:26.813472 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:26.813433    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:26.925698 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:26.925654    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:27.035655 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:27.035620    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:27.158301 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:27.158210    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:28.854827 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:28.854790    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:28.887269 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:28.887219    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq"]
Oct 31 21:57:29.040145 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.040103    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:29.504950 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.504902    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"]
Oct 31 21:57:29.505753 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.505720    4460 cpu_manager.go:414] "containerMap: removing container" podUID="6870d753-72cd-409c-947b-442703ac27d0" containerName="istio-proxy"
Oct 31 21:57:29.505753 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.505746    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6870d753-72cd-409c-947b-442703ac27d0" containerName="istio-proxy"
Oct 31 21:57:29.506032 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.506015    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="6870d753-72cd-409c-947b-442703ac27d0" containerName="istio-proxy"
Oct 31 21:57:29.510083 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.510061    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:57:29.527339 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.527302    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-4275b9-openshift-default-dockercfg-pbtv2\""
Oct 31 21:57:29.547167 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.547119    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"]
Oct 31 21:57:29.559776 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.559744    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:29.660546 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.660500    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1952bdcb-9771-45be-85ce-085a1c9df4f8-istiod-ca-cert\") pod \"e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") " pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:57:29.660855 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.660834    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1952bdcb-9771-45be-85ce-085a1c9df4f8-istio-podinfo\") pod \"e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") " pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:57:29.661068 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.661045    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbjd6\" (UniqueName: \"kubernetes.io/projected/1952bdcb-9771-45be-85ce-085a1c9df4f8-kube-api-access-bbjd6\") pod \"e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") " pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:57:29.661299 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.661192    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-istio-envoy\") pod \"e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") " pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:57:29.661438 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.661421    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-workload-socket\") pod \"e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") " pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:57:29.684109 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.684063    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-credential-socket\") pod \"e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") " pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:57:29.685553 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.685523    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-workload-certs\") pod \"e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") " pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:57:29.685889 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.685869    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-istio-data\") pod \"e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") " pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:57:29.686106 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.686085    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1952bdcb-9771-45be-85ce-085a1c9df4f8-istio-token\") pod \"e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") " pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:57:29.788836 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.788549    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1952bdcb-9771-45be-85ce-085a1c9df4f8-istiod-ca-cert\") pod \"e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") " pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:57:29.788836 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.788621    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1952bdcb-9771-45be-85ce-085a1c9df4f8-istio-podinfo\") pod \"e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") " pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:57:29.788836 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.788681    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbjd6\" (UniqueName: \"kubernetes.io/projected/1952bdcb-9771-45be-85ce-085a1c9df4f8-kube-api-access-bbjd6\") pod \"e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") " pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:57:29.788836 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.788711    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-istio-envoy\") pod \"e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") " pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:57:29.788836 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.788741    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-workload-socket\") pod \"e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") " pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:57:29.788836 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.788780    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-credential-socket\") pod \"e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") " pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:57:29.788836 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.788821    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-workload-certs\") pod \"e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") " pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:57:29.789391 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.788902    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-istio-data\") pod \"e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") " pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:57:29.789391 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.788984    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1952bdcb-9771-45be-85ce-085a1c9df4f8-istio-token\") pod \"e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") " pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:57:29.790269 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.790235    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1952bdcb-9771-45be-85ce-085a1c9df4f8-istiod-ca-cert\") pod \"e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") " pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:57:29.791836 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.790986    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-istio-data\") pod \"e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") " pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:57:29.791836 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.791232    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-workload-certs\") pod \"e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") " pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:57:29.792073 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.791867    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-workload-socket\") pod \"e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") " pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:57:29.792628 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.792604    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-credential-socket\") pod \"e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") " pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:57:29.797452 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.796784    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1952bdcb-9771-45be-85ce-085a1c9df4f8-istio-podinfo\") pod \"e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") " pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:57:29.800468 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.800427    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-istio-envoy\") pod \"e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") " pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:57:29.814361 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.814332    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1952bdcb-9771-45be-85ce-085a1c9df4f8-istio-token\") pod \"e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") " pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:57:29.816415 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.816393    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbjd6\" (UniqueName: \"kubernetes.io/projected/1952bdcb-9771-45be-85ce-085a1c9df4f8-kube-api-access-bbjd6\") pod \"e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") " pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:57:29.819841 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.819813    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:29.827692 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:29.827669    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:57:30.186574 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:30.186540    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"]
Oct 31 21:57:30.187416 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:57:30.187387    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1952bdcb_9771_45be_85ce_085a1c9df4f8.slice/crio-df2c502c0c5bbc47d7ed18d9e510b490ffa776ab4e1ab4b1749d72426ed83085 WatchSource:0}: Error finding container df2c502c0c5bbc47d7ed18d9e510b490ffa776ab4e1ab4b1749d72426ed83085: Status 404 returned error can't find the container with id df2c502c0c5bbc47d7ed18d9e510b490ffa776ab4e1ab4b1749d72426ed83085
Oct 31 21:57:30.190409 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:30.190375    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:57:30.190512 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:30.190456    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:57:30.190588 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:30.190510    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:57:30.736400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:30.736360    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:30.739139 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:30.739106    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns" event={"ID":"1952bdcb-9771-45be-85ce-085a1c9df4f8","Type":"ContainerStarted","Data":"9228743d8b134ad749810fcc221f0cd706c40b43977e9644126911a28ba76242"}
Oct 31 21:57:30.739279 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:30.739185    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns" event={"ID":"1952bdcb-9771-45be-85ce-085a1c9df4f8","Type":"ContainerStarted","Data":"df2c502c0c5bbc47d7ed18d9e510b490ffa776ab4e1ab4b1749d72426ed83085"}
Oct 31 21:57:30.826125 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:30.826070    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns" podStartSLOduration=1.826055995 podStartE2EDuration="1.826055995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:57:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:57:30.825333273 +0000 UTC m=+2490.741642713" watchObservedRunningTime="2026-10-31 21:57:30.826055995 +0000 UTC m=+2490.742365424"
Oct 31 21:57:30.828550 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:30.828523    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:57:31.834508 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:31.834471    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:57:32.668200 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:32.668161    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:32.758440 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:32.758409    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:57:32.759416 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:32.759389    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:57:33.157112 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:33.157075    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:33.266996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:33.266924    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:33.888092 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:33.888046    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq" podUID="7c695450-20d4-49f3-b9f4-a522980c1f83" containerName="payload-processing" containerID="cri-o://03a8a4ca52be07e1ea5ad2165f123b4abeb8891dca61f182e5e3c23e508323fb" gracePeriod=25
Oct 31 21:57:34.115063 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:34.115028    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:57:34.291868 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:34.278994    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq"
Oct 31 21:57:34.379348 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:34.379309    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7c695450-20d4-49f3-b9f4-a522980c1f83-kube-api-access-q5chl\" (UniqueName: \"kubernetes.io/projected/7c695450-20d4-49f3-b9f4-a522980c1f83-kube-api-access-q5chl\") pod \"7c695450-20d4-49f3-b9f4-a522980c1f83\" (UID: \"7c695450-20d4-49f3-b9f4-a522980c1f83\") "
Oct 31 21:57:34.379525 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:34.379477    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7c695450-20d4-49f3-b9f4-a522980c1f83-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7c695450-20d4-49f3-b9f4-a522980c1f83-routing-overlay-volume\") pod \"7c695450-20d4-49f3-b9f4-a522980c1f83\" (UID: \"7c695450-20d4-49f3-b9f4-a522980c1f83\") "
Oct 31 21:57:34.379598 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:34.379539    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7c695450-20d4-49f3-b9f4-a522980c1f83-provider-credentials\" (UniqueName: \"kubernetes.io/projected/7c695450-20d4-49f3-b9f4-a522980c1f83-provider-credentials\") pod \"7c695450-20d4-49f3-b9f4-a522980c1f83\" (UID: \"7c695450-20d4-49f3-b9f4-a522980c1f83\") "
Oct 31 21:57:34.379701 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:34.379679    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7c695450-20d4-49f3-b9f4-a522980c1f83-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c695450-20d4-49f3-b9f4-a522980c1f83-plugins-config-volume\") pod \"7c695450-20d4-49f3-b9f4-a522980c1f83\" (UID: \"7c695450-20d4-49f3-b9f4-a522980c1f83\") "
Oct 31 21:57:34.380020 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:34.379987    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c695450-20d4-49f3-b9f4-a522980c1f83-routing-overlay-volume" pod "7c695450-20d4-49f3-b9f4-a522980c1f83" (UID: "7c695450-20d4-49f3-b9f4-a522980c1f83"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:57:34.380190 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:34.380076    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c695450-20d4-49f3-b9f4-a522980c1f83-plugins-config-volume" pod "7c695450-20d4-49f3-b9f4-a522980c1f83" (UID: "7c695450-20d4-49f3-b9f4-a522980c1f83"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:57:34.380863 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:34.380831    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c695450-20d4-49f3-b9f4-a522980c1f83-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:57:34.381007 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:34.380870    4460 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7c695450-20d4-49f3-b9f4-a522980c1f83-routing-overlay-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:57:34.382322 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:34.382295    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c695450-20d4-49f3-b9f4-a522980c1f83-kube-api-access-q5chl" pod "7c695450-20d4-49f3-b9f4-a522980c1f83" (UID: "7c695450-20d4-49f3-b9f4-a522980c1f83"). InnerVolumeSpecName "kube-api-access-q5chl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:57:34.382448 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:34.382431    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c695450-20d4-49f3-b9f4-a522980c1f83-provider-credentials" pod "7c695450-20d4-49f3-b9f4-a522980c1f83" (UID: "7c695450-20d4-49f3-b9f4-a522980c1f83"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:57:34.482953 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:34.482905    4460 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/7c695450-20d4-49f3-b9f4-a522980c1f83-provider-credentials\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:57:34.482953 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:34.482949    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5chl\" (UniqueName: \"kubernetes.io/projected/7c695450-20d4-49f3-b9f4-a522980c1f83-kube-api-access-q5chl\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:57:34.691982 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:34.691882    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:34.775136 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:34.775102    4460 generic.go:356] "Generic (PLEG): container finished" podID="7c695450-20d4-49f3-b9f4-a522980c1f83" containerID="03a8a4ca52be07e1ea5ad2165f123b4abeb8891dca61f182e5e3c23e508323fb" exitCode=0
Oct 31 21:57:34.775304 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:34.775143    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq"
Oct 31 21:57:34.775304 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:34.775176    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq" event={"ID":"7c695450-20d4-49f3-b9f4-a522980c1f83","Type":"ContainerDied","Data":"03a8a4ca52be07e1ea5ad2165f123b4abeb8891dca61f182e5e3c23e508323fb"}
Oct 31 21:57:34.775304 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:34.775230    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq" event={"ID":"7c695450-20d4-49f3-b9f4-a522980c1f83","Type":"ContainerDied","Data":"84faa7c58ca1c8fc32d810117754c1966ffb7750756815969d9269a89efd7b78"}
Oct 31 21:57:34.775304 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:34.775253    4460 scope.go:122] "RemoveContainer" containerID="03a8a4ca52be07e1ea5ad2165f123b4abeb8891dca61f182e5e3c23e508323fb"
Oct 31 21:57:34.796333 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:34.795922    4460 scope.go:122] "RemoveContainer" containerID="03a8a4ca52be07e1ea5ad2165f123b4abeb8891dca61f182e5e3c23e508323fb"
Oct 31 21:57:34.796529 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:57:34.796357    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03a8a4ca52be07e1ea5ad2165f123b4abeb8891dca61f182e5e3c23e508323fb\": container with ID starting with 03a8a4ca52be07e1ea5ad2165f123b4abeb8891dca61f182e5e3c23e508323fb not found: ID does not exist" containerID="03a8a4ca52be07e1ea5ad2165f123b4abeb8891dca61f182e5e3c23e508323fb"
Oct 31 21:57:34.796529 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:34.796402    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03a8a4ca52be07e1ea5ad2165f123b4abeb8891dca61f182e5e3c23e508323fb"} err="failed to get container status \"03a8a4ca52be07e1ea5ad2165f123b4abeb8891dca61f182e5e3c23e508323fb\": rpc error: code = NotFound desc = could not find container \"03a8a4ca52be07e1ea5ad2165f123b4abeb8891dca61f182e5e3c23e508323fb\": container with ID starting with 03a8a4ca52be07e1ea5ad2165f123b4abeb8891dca61f182e5e3c23e508323fb not found: ID does not exist"
Oct 31 21:57:34.876401 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:34.871000    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq"]
Oct 31 21:57:34.929963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:34.912520    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-d44bjxbwq"]
Oct 31 21:57:35.113638 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:35.113606    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:35.224793 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:35.224755    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:36.226659 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:36.226627    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7c695450-20d4-49f3-b9f4-a522980c1f83" path="/var/lib/kubelet/pods/7c695450-20d4-49f3-b9f4-a522980c1f83/volumes"
Oct 31 21:57:36.632414 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:36.632380    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:38.444370 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:38.444330    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:40.228159 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:40.228124    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:42.063752 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:42.063709    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:43.828878 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:43.828828    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:44.084062 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:44.083951    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:45.654978 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:45.654927    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:47.388429 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:47.388390    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:48.685754 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:48.685718    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:49.242293 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:49.242255    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:49.993316 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:49.993279    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:51.022359 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:51.022320    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:51.454065 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:57:51.453983    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:57:52.849482 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:52.849442    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:53.411270 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:53.411234    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:54.582431 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:54.582397    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:54.692730 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:54.692688    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:54.860903 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:54.860793    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:55.062199 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:55.062160    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:55.179291 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:55.179190    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:55.318409 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:55.318374    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:55.432400 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:55.432322    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:55.662765 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:55.662723    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:55.794517 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:55.794474    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:56.293975 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:56.293924    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:56.406541 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:56.404703    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:56.843998 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:56.843927    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:57.172524 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:57.172431    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm"]
Oct 31 21:57:57.710971 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:57.710904    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:57:58.632719 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:57:58.632680    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:00.418232 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:00.418196    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:01.614327 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:01.614291    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:01.728249 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:01.728205    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:02.173177 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:02.173135    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm" podUID="54c25d6d-76c7-422d-b0a4-9bc3281dc818" containerName="payload-processing" containerID="cri-o://4b2e085faaba9164524e2a656f5d105a05f620aa6192366b4bd94db072df4978" gracePeriod=25
Oct 31 21:58:02.248402 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:02.248371    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:02.506589 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:02.506566    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm"
Oct 31 21:58:02.587027 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:02.586996    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/54c25d6d-76c7-422d-b0a4-9bc3281dc818-kube-api-access-kprhv\" (UniqueName: \"kubernetes.io/projected/54c25d6d-76c7-422d-b0a4-9bc3281dc818-kube-api-access-kprhv\") pod \"54c25d6d-76c7-422d-b0a4-9bc3281dc818\" (UID: \"54c25d6d-76c7-422d-b0a4-9bc3281dc818\") "
Oct 31 21:58:02.587265 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:02.587113    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/54c25d6d-76c7-422d-b0a4-9bc3281dc818-provider-credentials\" (UniqueName: \"kubernetes.io/projected/54c25d6d-76c7-422d-b0a4-9bc3281dc818-provider-credentials\") pod \"54c25d6d-76c7-422d-b0a4-9bc3281dc818\" (UID: \"54c25d6d-76c7-422d-b0a4-9bc3281dc818\") "
Oct 31 21:58:02.587367 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:02.587309    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/54c25d6d-76c7-422d-b0a4-9bc3281dc818-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/54c25d6d-76c7-422d-b0a4-9bc3281dc818-routing-overlay-volume\") pod \"54c25d6d-76c7-422d-b0a4-9bc3281dc818\" (UID: \"54c25d6d-76c7-422d-b0a4-9bc3281dc818\") "
Oct 31 21:58:02.587455 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:02.587394    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/54c25d6d-76c7-422d-b0a4-9bc3281dc818-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54c25d6d-76c7-422d-b0a4-9bc3281dc818-plugins-config-volume\") pod \"54c25d6d-76c7-422d-b0a4-9bc3281dc818\" (UID: \"54c25d6d-76c7-422d-b0a4-9bc3281dc818\") "
Oct 31 21:58:02.588108 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:02.587916    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54c25d6d-76c7-422d-b0a4-9bc3281dc818-routing-overlay-volume" pod "54c25d6d-76c7-422d-b0a4-9bc3281dc818" (UID: "54c25d6d-76c7-422d-b0a4-9bc3281dc818"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:58:02.588498 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:02.588465    4460 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/54c25d6d-76c7-422d-b0a4-9bc3281dc818-routing-overlay-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:58:02.588498 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:02.588452    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54c25d6d-76c7-422d-b0a4-9bc3281dc818-plugins-config-volume" pod "54c25d6d-76c7-422d-b0a4-9bc3281dc818" (UID: "54c25d6d-76c7-422d-b0a4-9bc3281dc818"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:58:02.590697 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:02.590671    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54c25d6d-76c7-422d-b0a4-9bc3281dc818-kube-api-access-kprhv" pod "54c25d6d-76c7-422d-b0a4-9bc3281dc818" (UID: "54c25d6d-76c7-422d-b0a4-9bc3281dc818"). InnerVolumeSpecName "kube-api-access-kprhv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:58:02.590813 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:02.590714    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54c25d6d-76c7-422d-b0a4-9bc3281dc818-provider-credentials" pod "54c25d6d-76c7-422d-b0a4-9bc3281dc818" (UID: "54c25d6d-76c7-422d-b0a4-9bc3281dc818"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:58:02.689715 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:02.689680    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54c25d6d-76c7-422d-b0a4-9bc3281dc818-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:58:02.689715 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:02.689710    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kprhv\" (UniqueName: \"kubernetes.io/projected/54c25d6d-76c7-422d-b0a4-9bc3281dc818-kube-api-access-kprhv\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:58:02.689715 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:02.689720    4460 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/54c25d6d-76c7-422d-b0a4-9bc3281dc818-provider-credentials\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:58:03.030657 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:03.030623    4460 generic.go:356] "Generic (PLEG): container finished" podID="54c25d6d-76c7-422d-b0a4-9bc3281dc818" containerID="4b2e085faaba9164524e2a656f5d105a05f620aa6192366b4bd94db072df4978" exitCode=0
Oct 31 21:58:03.030839 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:03.030667    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm"
Oct 31 21:58:03.030839 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:03.030709    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm" event={"ID":"54c25d6d-76c7-422d-b0a4-9bc3281dc818","Type":"ContainerDied","Data":"4b2e085faaba9164524e2a656f5d105a05f620aa6192366b4bd94db072df4978"}
Oct 31 21:58:03.030839 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:03.030768    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm" event={"ID":"54c25d6d-76c7-422d-b0a4-9bc3281dc818","Type":"ContainerDied","Data":"c29028491493632993bebc2dd6ea376659bba1d9340fd23a587c332df03ea1dd"}
Oct 31 21:58:03.030839 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:03.030796    4460 scope.go:122] "RemoveContainer" containerID="4b2e085faaba9164524e2a656f5d105a05f620aa6192366b4bd94db072df4978"
Oct 31 21:58:03.045460 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:03.045440    4460 scope.go:122] "RemoveContainer" containerID="4b2e085faaba9164524e2a656f5d105a05f620aa6192366b4bd94db072df4978"
Oct 31 21:58:03.045783 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:58:03.045757    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b2e085faaba9164524e2a656f5d105a05f620aa6192366b4bd94db072df4978\": container with ID starting with 4b2e085faaba9164524e2a656f5d105a05f620aa6192366b4bd94db072df4978 not found: ID does not exist" containerID="4b2e085faaba9164524e2a656f5d105a05f620aa6192366b4bd94db072df4978"
Oct 31 21:58:03.045843 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:03.045791    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b2e085faaba9164524e2a656f5d105a05f620aa6192366b4bd94db072df4978"} err="failed to get container status \"4b2e085faaba9164524e2a656f5d105a05f620aa6192366b4bd94db072df4978\": rpc error: code = NotFound desc = could not find container \"4b2e085faaba9164524e2a656f5d105a05f620aa6192366b4bd94db072df4978\": container with ID starting with 4b2e085faaba9164524e2a656f5d105a05f620aa6192366b4bd94db072df4978 not found: ID does not exist"
Oct 31 21:58:03.088136 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:03.088113    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm"]
Oct 31 21:58:03.101417 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:03.101393    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-6c85hzhjm"]
Oct 31 21:58:04.036764 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:04.036728    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:04.150690 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:04.150652    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:04.230024 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:04.229984    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="54c25d6d-76c7-422d-b0a4-9bc3281dc818" path="/var/lib/kubelet/pods/54c25d6d-76c7-422d-b0a4-9bc3281dc818/volumes"
Oct 31 21:58:04.259581 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:04.259524    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:05.104953 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:05.104918    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:58:05.961964 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:05.961025    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:07.731426 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:07.731388    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:08.577922 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:08.577891    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:08.691709 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:08.691670    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:09.241205 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:09.241156    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:09.514588 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:09.514556    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:09.634752 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:09.634719    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:09.744344 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:09.744304    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:11.203865 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:11.203825    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:11.738660 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:11.738614    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:11.966059 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:11.966026    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq"]
Oct 31 21:58:11.966865 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:11.966847    4460 cpu_manager.go:414] "containerMap: removing container" podUID="54c25d6d-76c7-422d-b0a4-9bc3281dc818" containerName="payload-processing"
Oct 31 21:58:11.966913 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:11.966867    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="54c25d6d-76c7-422d-b0a4-9bc3281dc818" containerName="payload-processing"
Oct 31 21:58:11.966969 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:11.966927    4460 cpu_manager.go:414] "containerMap: removing container" podUID="7c695450-20d4-49f3-b9f4-a522980c1f83" containerName="payload-processing"
Oct 31 21:58:11.966969 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:11.966933    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c695450-20d4-49f3-b9f4-a522980c1f83" containerName="payload-processing"
Oct 31 21:58:11.967120 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:11.967107    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="7c695450-20d4-49f3-b9f4-a522980c1f83" containerName="payload-processing"
Oct 31 21:58:11.967159 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:11.967128    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="54c25d6d-76c7-422d-b0a4-9bc3281dc818" containerName="payload-processing"
Oct 31 21:58:11.972556 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:11.972533    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq"
Oct 31 21:58:11.987285 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:11.987262    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq"]
Oct 31 21:58:11.996999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:11.996909    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-c34cbd\"/\"agc-cred-3d39da95e8af\""
Oct 31 21:58:11.997173 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:11.997157    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-c34cbd\"/\"openshift-service-ca.crt\""
Oct 31 21:58:11.997255 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:11.997212    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-c34cbd\"/\"payload-processing-external-model-e2e--bb6f7797-dockercfg-6p5zs\""
Oct 31 21:58:11.997338 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:11.997322    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-c34cbd\"/\"kube-root-ca.crt\""
Oct 31 21:58:12.002972 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:12.002929    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-c34cbd\"/\"payload-processing-external-model-plugins-e2e-extmod-b-c34cbd\""
Oct 31 21:58:12.002972 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:12.002960    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-c34cbd\"/\"routing-overlay\""
Oct 31 21:58:12.099889 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:12.099845    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgs28\" (UniqueName: \"kubernetes.io/projected/726341b9-8504-4a86-86cf-24792d452681-kube-api-access-cgs28\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq\" (UID: \"726341b9-8504-4a86-86cf-24792d452681\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq"
Oct 31 21:58:12.100102 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:12.100035    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/726341b9-8504-4a86-86cf-24792d452681-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq\" (UID: \"726341b9-8504-4a86-86cf-24792d452681\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq"
Oct 31 21:58:12.100197 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:12.100119    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/726341b9-8504-4a86-86cf-24792d452681-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq\" (UID: \"726341b9-8504-4a86-86cf-24792d452681\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq"
Oct 31 21:58:12.100197 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:12.100161    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/726341b9-8504-4a86-86cf-24792d452681-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq\" (UID: \"726341b9-8504-4a86-86cf-24792d452681\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq"
Oct 31 21:58:12.201855 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:12.201818    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/726341b9-8504-4a86-86cf-24792d452681-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq\" (UID: \"726341b9-8504-4a86-86cf-24792d452681\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq"
Oct 31 21:58:12.202061 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:12.201881    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/726341b9-8504-4a86-86cf-24792d452681-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq\" (UID: \"726341b9-8504-4a86-86cf-24792d452681\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq"
Oct 31 21:58:12.202061 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:12.201912    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/726341b9-8504-4a86-86cf-24792d452681-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq\" (UID: \"726341b9-8504-4a86-86cf-24792d452681\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq"
Oct 31 21:58:12.202232 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:12.202202    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgs28\" (UniqueName: \"kubernetes.io/projected/726341b9-8504-4a86-86cf-24792d452681-kube-api-access-cgs28\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq\" (UID: \"726341b9-8504-4a86-86cf-24792d452681\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq"
Oct 31 21:58:12.202736 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:12.202708    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/726341b9-8504-4a86-86cf-24792d452681-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq\" (UID: \"726341b9-8504-4a86-86cf-24792d452681\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq"
Oct 31 21:58:12.202849 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:12.202803    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/726341b9-8504-4a86-86cf-24792d452681-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq\" (UID: \"726341b9-8504-4a86-86cf-24792d452681\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq"
Oct 31 21:58:12.204853 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:12.204815    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/726341b9-8504-4a86-86cf-24792d452681-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq\" (UID: \"726341b9-8504-4a86-86cf-24792d452681\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq"
Oct 31 21:58:12.228131 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:12.228100    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgs28\" (UniqueName: \"kubernetes.io/projected/726341b9-8504-4a86-86cf-24792d452681-kube-api-access-cgs28\") pod \"payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq\" (UID: \"726341b9-8504-4a86-86cf-24792d452681\") " pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq"
Oct 31 21:58:12.288732 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:12.288701    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq"
Oct 31 21:58:12.600581 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:58:12.600550    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod726341b9_8504_4a86_86cf_24792d452681.slice/crio-f7c2ad7953c010c2e1487f3356789a8f1702f253f123ec42dc5d0a852d7015f1 WatchSource:0}: Error finding container f7c2ad7953c010c2e1487f3356789a8f1702f253f123ec42dc5d0a852d7015f1: Status 404 returned error can't find the container with id f7c2ad7953c010c2e1487f3356789a8f1702f253f123ec42dc5d0a852d7015f1
Oct 31 21:58:12.601497 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:12.601470    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq"]
Oct 31 21:58:13.138534 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:13.138487    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq" event={"ID":"726341b9-8504-4a86-86cf-24792d452681","Type":"ContainerStarted","Data":"77dc181232c147a96dfc369753c1fafe9014022f7c16ed2e91b95d9a793bee7a"}
Oct 31 21:58:13.138735 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:13.138559    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq" event={"ID":"726341b9-8504-4a86-86cf-24792d452681","Type":"ContainerStarted","Data":"f7c2ad7953c010c2e1487f3356789a8f1702f253f123ec42dc5d0a852d7015f1"}
Oct 31 21:58:13.138735 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:13.138588    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq"
Oct 31 21:58:13.186351 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:13.186303    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq" podStartSLOduration=2.186288478 podStartE2EDuration="2.186288478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:58:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:58:13.183330741 +0000 UTC m=+2533.099640172" watchObservedRunningTime="2026-10-31 21:58:13.186288478 +0000 UTC m=+2533.102597944"
Oct 31 21:58:13.588545 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:13.588499    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:13.702237 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:13.702199    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:13.811033 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:13.810997    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:15.544453 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:15.544415    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:15.884682 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:15.884603    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:16.007275 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:16.007236    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:16.116572 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:16.116533    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:17.309369 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:17.309328    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:18.141514 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:18.141476    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:18.267036 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:18.267003    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:19.109249 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:19.109213    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:20.844587 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:20.844551    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:21.454072 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:58:21.454024    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:58:22.391444 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:22.391413    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:22.514739 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:22.514693    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:22.629642 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:22.629608    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:22.843450 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:22.843407    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:22.959313 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:22.959268    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:23.068999 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:23.068966    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:23.277844 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:23.277811    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:23.932557 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:23.932523    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:24.060682 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:24.060649    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:24.148950 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:24.148909    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq"
Oct 31 21:58:24.640452 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:24.640420    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:25.317516 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:25.317479    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:25.570743 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:25.570665    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k"]
Oct 31 21:58:25.575459 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:25.575439    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k"
Oct 31 21:58:25.594956 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:25.592304    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-fac043\"/\"agc-cred-7cc44fcedad2\""
Oct 31 21:58:25.594956 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:25.594247    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k"]
Oct 31 21:58:25.609172 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:25.608250    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-fac043\"/\"routing-overlay\""
Oct 31 21:58:25.609172 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:25.608380    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-fac043\"/\"payload-processing-external-model-e2e--2792e8e2-dockercfg-49svf\""
Oct 31 21:58:25.609172 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:25.609044    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-fac043\"/\"payload-processing-external-model-plugins-e2e-extmod-a-fac043\""
Oct 31 21:58:25.609172 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:25.609117    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-fac043\"/\"kube-root-ca.crt\""
Oct 31 21:58:25.609172 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:25.609142    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-fac043\"/\"openshift-service-ca.crt\""
Oct 31 21:58:25.775963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:25.766774    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a792d40-6ac4-4859-ab09-71aceed3248f-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k\" (UID: \"2a792d40-6ac4-4859-ab09-71aceed3248f\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k"
Oct 31 21:58:25.775963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:25.766997    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2a792d40-6ac4-4859-ab09-71aceed3248f-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k\" (UID: \"2a792d40-6ac4-4859-ab09-71aceed3248f\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k"
Oct 31 21:58:25.775963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:25.767037    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nblsw\" (UniqueName: \"kubernetes.io/projected/2a792d40-6ac4-4859-ab09-71aceed3248f-kube-api-access-nblsw\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k\" (UID: \"2a792d40-6ac4-4859-ab09-71aceed3248f\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k"
Oct 31 21:58:25.775963 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:25.767122    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/2a792d40-6ac4-4859-ab09-71aceed3248f-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k\" (UID: \"2a792d40-6ac4-4859-ab09-71aceed3248f\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k"
Oct 31 21:58:25.886718 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:25.886539    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/2a792d40-6ac4-4859-ab09-71aceed3248f-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k\" (UID: \"2a792d40-6ac4-4859-ab09-71aceed3248f\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k"
Oct 31 21:58:25.886919 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:25.886746    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a792d40-6ac4-4859-ab09-71aceed3248f-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k\" (UID: \"2a792d40-6ac4-4859-ab09-71aceed3248f\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k"
Oct 31 21:58:25.886919 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:25.886899    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2a792d40-6ac4-4859-ab09-71aceed3248f-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k\" (UID: \"2a792d40-6ac4-4859-ab09-71aceed3248f\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k"
Oct 31 21:58:25.887107 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:25.886931    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nblsw\" (UniqueName: \"kubernetes.io/projected/2a792d40-6ac4-4859-ab09-71aceed3248f-kube-api-access-nblsw\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k\" (UID: \"2a792d40-6ac4-4859-ab09-71aceed3248f\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k"
Oct 31 21:58:25.888805 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:25.888765    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2a792d40-6ac4-4859-ab09-71aceed3248f-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k\" (UID: \"2a792d40-6ac4-4859-ab09-71aceed3248f\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k"
Oct 31 21:58:25.889312 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:25.889287    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a792d40-6ac4-4859-ab09-71aceed3248f-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k\" (UID: \"2a792d40-6ac4-4859-ab09-71aceed3248f\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k"
Oct 31 21:58:25.890220 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:25.890199    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/2a792d40-6ac4-4859-ab09-71aceed3248f-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k\" (UID: \"2a792d40-6ac4-4859-ab09-71aceed3248f\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k"
Oct 31 21:58:25.912134 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:25.912109    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nblsw\" (UniqueName: \"kubernetes.io/projected/2a792d40-6ac4-4859-ab09-71aceed3248f-kube-api-access-nblsw\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k\" (UID: \"2a792d40-6ac4-4859-ab09-71aceed3248f\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k"
Oct 31 21:58:26.192534 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:26.192451    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k"
Oct 31 21:58:26.503671 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:26.503642    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k"]
Oct 31 21:58:26.504240 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:58:26.504217    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a792d40_6ac4_4859_ab09_71aceed3248f.slice/crio-c836a7210ca292763676396daacbe358acd550f1ee4bfc3c1c0e08e381b62353 WatchSource:0}: Error finding container c836a7210ca292763676396daacbe358acd550f1ee4bfc3c1c0e08e381b62353: Status 404 returned error can't find the container with id c836a7210ca292763676396daacbe358acd550f1ee4bfc3c1c0e08e381b62353
Oct 31 21:58:26.522149 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:26.522116    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:27.282877 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:27.282837    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k" event={"ID":"2a792d40-6ac4-4859-ab09-71aceed3248f","Type":"ContainerStarted","Data":"7ed8d98498db09c29374a79f06bcc1aac50045f76b3bf4c85d83a06b24c79462"}
Oct 31 21:58:27.283103 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:27.282893    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k" event={"ID":"2a792d40-6ac4-4859-ab09-71aceed3248f","Type":"ContainerStarted","Data":"c836a7210ca292763676396daacbe358acd550f1ee4bfc3c1c0e08e381b62353"}
Oct 31 21:58:27.283103 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:27.282922    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k"
Oct 31 21:58:27.328530 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:27.328480    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k" podStartSLOduration=2.328466819 podStartE2EDuration="2.328466819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:58:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:58:27.327096278 +0000 UTC m=+2547.243405720" watchObservedRunningTime="2026-10-31 21:58:27.328466819 +0000 UTC m=+2547.244776248"
Oct 31 21:58:28.257739 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:28.257707    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:30.047408 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:30.047370    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:31.831789 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:31.831750    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:33.650544 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:33.650490    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:35.384702 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:35.384664    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:37.237968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:37.234491    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:38.293749 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:38.293716    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k"
Oct 31 21:58:39.054073 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:39.054037    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:39.175028 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:39.174989    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:40.945776 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:40.945730    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:41.549896 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:41.549856    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:42.733420 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:42.733381    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:44.549980 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:44.549921    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:46.334494 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:46.334463    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:46.491086 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:46.491004    4460 generic.go:356] "Generic (PLEG): container finished" podID="1952bdcb-9771-45be-85ce-085a1c9df4f8" containerID="9228743d8b134ad749810fcc221f0cd706c40b43977e9644126911a28ba76242" exitCode=0
Oct 31 21:58:46.491241 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:46.491088    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns" event={"ID":"1952bdcb-9771-45be-85ce-085a1c9df4f8","Type":"ContainerDied","Data":"9228743d8b134ad749810fcc221f0cd706c40b43977e9644126911a28ba76242"}
Oct 31 21:58:46.491640 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:46.491624    4460 scope.go:122] "RemoveContainer" containerID="9228743d8b134ad749810fcc221f0cd706c40b43977e9644126911a28ba76242"
Oct 31 21:58:46.493642 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:46.493608    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:58:46.493765 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:46.493701    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:58:46.493765 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:46.493756    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 21:58:47.048436 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:47.048396    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:47.516260 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:47.515146    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns" event={"ID":"1952bdcb-9771-45be-85ce-085a1c9df4f8","Type":"ContainerStarted","Data":"5be606e49fcbafc179b3b0af46223d6eaf056c8f0c0a59f45aa1508f00c19326"}
Oct 31 21:58:47.758893 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:47.758843    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:58:47.841261 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:47.841113    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:58:47.842776 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:47.842744    4460 patch_prober.go:29] interesting pod/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused" start-of-body=
Oct 31 21:58:47.843019 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:47.842992    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns" podUID="1952bdcb-9771-45be-85ce-085a1c9df4f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused"
Oct 31 21:58:48.248108 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:48.248019    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:48.801913 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:48.801875    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:48.830358 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:48.830321    4460 patch_prober.go:29] interesting pod/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused" start-of-body=
Oct 31 21:58:48.830649 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:48.830619    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns" podUID="1952bdcb-9771-45be-85ce-085a1c9df4f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused"
Oct 31 21:58:49.845040 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:49.845003    4460 patch_prober.go:29] interesting pod/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused" start-of-body=
Oct 31 21:58:49.845891 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:49.845092    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns" podUID="1952bdcb-9771-45be-85ce-085a1c9df4f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused"
Oct 31 21:58:50.046592 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:50.046548    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:50.155741 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:50.155640    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:50.835965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:50.831357    4460 patch_prober.go:29] interesting pod/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused" start-of-body=
Oct 31 21:58:50.835965 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:50.831419    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns" podUID="1952bdcb-9771-45be-85ce-085a1c9df4f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused"
Oct 31 21:58:51.472532 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:58:51.472498    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:58:51.845397 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:51.845362    4460 patch_prober.go:29] interesting pod/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused" start-of-body=
Oct 31 21:58:51.845612 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:51.845567    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns" podUID="1952bdcb-9771-45be-85ce-085a1c9df4f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.179:15021/healthz/ready\": dial tcp 10.133.0.179:15021: connect: connection refused"
Oct 31 21:58:51.847679 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:51.847645    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:52.832935 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:52.832905    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:58:52.833903 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:52.833884    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 21:58:53.096835 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:53.096730    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq"]
Oct 31 21:58:53.100524 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:53.100466    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:53.219593 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:53.219555    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:53.652446 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:53.652406    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:53.765409 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:53.765364    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:55.548315 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:55.548279    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:55.684996 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:55.679465    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:57.094962 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:57.091503    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:57.355740 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:57.355651    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:58:58.097802 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:58.097753    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq" podUID="726341b9-8504-4a86-86cf-24792d452681" containerName="payload-processing" containerID="cri-o://77dc181232c147a96dfc369753c1fafe9014022f7c16ed2e91b95d9a793bee7a" gracePeriod=25
Oct 31 21:58:58.390049 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:58.390022    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq"
Oct 31 21:58:58.527426 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:58.527384    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/726341b9-8504-4a86-86cf-24792d452681-provider-credentials\" (UniqueName: \"kubernetes.io/projected/726341b9-8504-4a86-86cf-24792d452681-provider-credentials\") pod \"726341b9-8504-4a86-86cf-24792d452681\" (UID: \"726341b9-8504-4a86-86cf-24792d452681\") "
Oct 31 21:58:58.527623 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:58.527487    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/726341b9-8504-4a86-86cf-24792d452681-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/726341b9-8504-4a86-86cf-24792d452681-plugins-config-volume\") pod \"726341b9-8504-4a86-86cf-24792d452681\" (UID: \"726341b9-8504-4a86-86cf-24792d452681\") "
Oct 31 21:58:58.527694 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:58.527636    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/726341b9-8504-4a86-86cf-24792d452681-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/726341b9-8504-4a86-86cf-24792d452681-routing-overlay-volume\") pod \"726341b9-8504-4a86-86cf-24792d452681\" (UID: \"726341b9-8504-4a86-86cf-24792d452681\") "
Oct 31 21:58:58.527694 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:58.527670    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/726341b9-8504-4a86-86cf-24792d452681-kube-api-access-cgs28\" (UniqueName: \"kubernetes.io/projected/726341b9-8504-4a86-86cf-24792d452681-kube-api-access-cgs28\") pod \"726341b9-8504-4a86-86cf-24792d452681\" (UID: \"726341b9-8504-4a86-86cf-24792d452681\") "
Oct 31 21:58:58.527983 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:58.527924    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/726341b9-8504-4a86-86cf-24792d452681-plugins-config-volume" pod "726341b9-8504-4a86-86cf-24792d452681" (UID: "726341b9-8504-4a86-86cf-24792d452681"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:58:58.528070 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:58.527978    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/726341b9-8504-4a86-86cf-24792d452681-routing-overlay-volume" pod "726341b9-8504-4a86-86cf-24792d452681" (UID: "726341b9-8504-4a86-86cf-24792d452681"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:58:58.528311 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:58.528295    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/726341b9-8504-4a86-86cf-24792d452681-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:58:58.528352 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:58.528316    4460 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/726341b9-8504-4a86-86cf-24792d452681-routing-overlay-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:58:58.529687 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:58.529660    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/726341b9-8504-4a86-86cf-24792d452681-provider-credentials" pod "726341b9-8504-4a86-86cf-24792d452681" (UID: "726341b9-8504-4a86-86cf-24792d452681"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:58:58.529803 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:58.529787    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/726341b9-8504-4a86-86cf-24792d452681-kube-api-access-cgs28" pod "726341b9-8504-4a86-86cf-24792d452681" (UID: "726341b9-8504-4a86-86cf-24792d452681"). InnerVolumeSpecName "kube-api-access-cgs28". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:58:58.629836 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:58.629757    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgs28\" (UniqueName: \"kubernetes.io/projected/726341b9-8504-4a86-86cf-24792d452681-kube-api-access-cgs28\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:58:58.629836 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:58.629790    4460 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/726341b9-8504-4a86-86cf-24792d452681-provider-credentials\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:58:58.719735 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:58.719694    4460 generic.go:356] "Generic (PLEG): container finished" podID="726341b9-8504-4a86-86cf-24792d452681" containerID="77dc181232c147a96dfc369753c1fafe9014022f7c16ed2e91b95d9a793bee7a" exitCode=0
Oct 31 21:58:58.719953 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:58.719748    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq"
Oct 31 21:58:58.719953 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:58.719774    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq" event={"ID":"726341b9-8504-4a86-86cf-24792d452681","Type":"ContainerDied","Data":"77dc181232c147a96dfc369753c1fafe9014022f7c16ed2e91b95d9a793bee7a"}
Oct 31 21:58:58.719953 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:58.719847    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq" event={"ID":"726341b9-8504-4a86-86cf-24792d452681","Type":"ContainerDied","Data":"f7c2ad7953c010c2e1487f3356789a8f1702f253f123ec42dc5d0a852d7015f1"}
Oct 31 21:58:58.719953 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:58.719885    4460 scope.go:122] "RemoveContainer" containerID="77dc181232c147a96dfc369753c1fafe9014022f7c16ed2e91b95d9a793bee7a"
Oct 31 21:58:58.735110 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:58.735088    4460 scope.go:122] "RemoveContainer" containerID="77dc181232c147a96dfc369753c1fafe9014022f7c16ed2e91b95d9a793bee7a"
Oct 31 21:58:58.735381 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:58:58.735364    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77dc181232c147a96dfc369753c1fafe9014022f7c16ed2e91b95d9a793bee7a\": container with ID starting with 77dc181232c147a96dfc369753c1fafe9014022f7c16ed2e91b95d9a793bee7a not found: ID does not exist" containerID="77dc181232c147a96dfc369753c1fafe9014022f7c16ed2e91b95d9a793bee7a"
Oct 31 21:58:58.735427 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:58.735391    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77dc181232c147a96dfc369753c1fafe9014022f7c16ed2e91b95d9a793bee7a"} err="failed to get container status \"77dc181232c147a96dfc369753c1fafe9014022f7c16ed2e91b95d9a793bee7a\": rpc error: code = NotFound desc = could not find container \"77dc181232c147a96dfc369753c1fafe9014022f7c16ed2e91b95d9a793bee7a\": container with ID starting with 77dc181232c147a96dfc369753c1fafe9014022f7c16ed2e91b95d9a793bee7a not found: ID does not exist"
Oct 31 21:58:58.779691 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:58.779664    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq"]
Oct 31 21:58:58.792372 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:58.792348    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-c34cbd/payload-processing-external-model-e2e-extmod-b-c34cbd-7d45f8rdq"]
Oct 31 21:58:59.085242 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:58:59.085195    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:00.054705 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:00.054664    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:00.227690 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:00.227657    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="726341b9-8504-4a86-86cf-24792d452681" path="/var/lib/kubelet/pods/726341b9-8504-4a86-86cf-24792d452681/volumes"
Oct 31 21:59:00.275459 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:00.275422    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:00.385798 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:00.385704    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:00.493872 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:00.493833    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:00.916120 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:00.916076    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:01.406396 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:01.406365    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:01.871818 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:01.871775    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:01.980818 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:01.980769    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:02.689157 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:02.689122    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:03.284881 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:03.284844    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:03.392108 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:03.392072    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:04.531991 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:04.531930    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:05.227152 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:05.227116    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:05.351979 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:05.351929    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:06.284577 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:06.284541    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:06.967310 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:06.967258    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:07.007321 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:07.007285    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:59:07.082386 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:07.082347    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:07.149766 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:07.149722    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-a5affe-78c59c567-c6hdt"]
Oct 31 21:59:07.150551 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:07.150535    4460 cpu_manager.go:414] "containerMap: removing container" podUID="726341b9-8504-4a86-86cf-24792d452681" containerName="payload-processing"
Oct 31 21:59:07.150608 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:07.150554    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="726341b9-8504-4a86-86cf-24792d452681" containerName="payload-processing"
Oct 31 21:59:07.150675 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:07.150664    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="726341b9-8504-4a86-86cf-24792d452681" containerName="payload-processing"
Oct 31 21:59:07.156079 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:07.156058    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a5affe-78c59c567-c6hdt"
Oct 31 21:59:07.172880 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:07.172846    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-a5affe\""
Oct 31 21:59:07.172880 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:07.172866    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-a5affe-dockercfg-82rzx\""
Oct 31 21:59:07.175872 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:07.175848    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-a5affe-78c59c567-c6hdt"]
Oct 31 21:59:07.331968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:07.331659    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-a5affe-78c59c567-c6hdt\" (UID: \"8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a5affe-78c59c567-c6hdt"
Oct 31 21:59:07.331968 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:07.331784    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwvj4\" (UniqueName: \"kubernetes.io/projected/8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1-kube-api-access-rwvj4\") pod \"payload-pre-processing-e2e-extmod-b-a5affe-78c59c567-c6hdt\" (UID: \"8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a5affe-78c59c567-c6hdt"
Oct 31 21:59:07.433019 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:07.432980    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-a5affe-78c59c567-c6hdt\" (UID: \"8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a5affe-78c59c567-c6hdt"
Oct 31 21:59:07.433239 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:07.433090    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwvj4\" (UniqueName: \"kubernetes.io/projected/8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1-kube-api-access-rwvj4\") pod \"payload-pre-processing-e2e-extmod-b-a5affe-78c59c567-c6hdt\" (UID: \"8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a5affe-78c59c567-c6hdt"
Oct 31 21:59:07.434053 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:07.434023    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-a5affe-78c59c567-c6hdt\" (UID: \"8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a5affe-78c59c567-c6hdt"
Oct 31 21:59:07.458431 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:07.458402    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwvj4\" (UniqueName: \"kubernetes.io/projected/8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1-kube-api-access-rwvj4\") pod \"payload-pre-processing-e2e-extmod-b-a5affe-78c59c567-c6hdt\" (UID: \"8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a5affe-78c59c567-c6hdt"
Oct 31 21:59:07.473312 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:07.473283    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a5affe-78c59c567-c6hdt"
Oct 31 21:59:07.621765 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:07.621680    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:07.734450 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:07.734418    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:07.786261 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:07.786230    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-a5affe-78c59c567-c6hdt"]
Oct 31 21:59:07.787010 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:59:07.786986    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b588e9e_1de6_4f7a_b7fd_2f98cde2edf1.slice/crio-421b614a416e51169d6ceb4ef1da87f1fe9df32cd4495b92be57efecaab345d7 WatchSource:0}: Error finding container 421b614a416e51169d6ceb4ef1da87f1fe9df32cd4495b92be57efecaab345d7: Status 404 returned error can't find the container with id 421b614a416e51169d6ceb4ef1da87f1fe9df32cd4495b92be57efecaab345d7
Oct 31 21:59:07.827438 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:07.827402    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a5affe-78c59c567-c6hdt" event={"ID":"8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1","Type":"ContainerStarted","Data":"421b614a416e51169d6ceb4ef1da87f1fe9df32cd4495b92be57efecaab345d7"}
Oct 31 21:59:07.865165 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:07.865117    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:08.036779 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:08.036745    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:08.164981 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:08.162864    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:08.286009 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:08.285794    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:08.455616 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:08.455577    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:08.837018 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:08.836985    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a5affe-78c59c567-c6hdt" event={"ID":"8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1","Type":"ContainerStarted","Data":"4574ea54ca431d88627d1a74c901ebbed08beafcd9a87dc16f9176aa627dd7f7"}
Oct 31 21:59:08.837171 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:08.837154    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a5affe-78c59c567-c6hdt"
Oct 31 21:59:08.889301 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:08.889248    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a5affe-78c59c567-c6hdt" podStartSLOduration=1.889233639 podStartE2EDuration="1.889233639s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:59:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:59:08.885411744 +0000 UTC m=+2588.801721174" watchObservedRunningTime="2026-10-31 21:59:08.889233639 +0000 UTC m=+2588.805543086"
Oct 31 21:59:09.058730 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:09.058682    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:09.168791 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:09.168693    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:10.078914 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:10.078879    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:10.605524 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:10.605484    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:10.667683 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:10.667645    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"]
Oct 31 21:59:10.667970 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:10.667923    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m" podUID="845e8f3e-1896-4417-bfe3-5d1638925a57" containerName="istio-proxy" containerID="cri-o://27c3d84586b0f096b6eed435b3db4a574ffba34a75a0abd502e9f8cdf370bbbd" gracePeriod=30
Oct 31 21:59:11.583196 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:11.583159    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:11.745824 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:11.745745    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:11.856422 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:11.856325    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:12.030309 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:12.030264    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:12.165007 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:12.162497    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:13.505740 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:13.505701    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:15.266383 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:15.266350    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:15.400546 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:15.400508    4460 patch_prober.go:29] interesting pod/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 21:59:15.400730 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:15.400571    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m" podUID="845e8f3e-1896-4417-bfe3-5d1638925a57" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 21:59:15.894699 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:15.894654    4460 generic.go:356] "Generic (PLEG): container finished" podID="845e8f3e-1896-4417-bfe3-5d1638925a57" containerID="27c3d84586b0f096b6eed435b3db4a574ffba34a75a0abd502e9f8cdf370bbbd" exitCode=0
Oct 31 21:59:15.894889 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:15.894713    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m" event={"ID":"845e8f3e-1896-4417-bfe3-5d1638925a57","Type":"ContainerDied","Data":"27c3d84586b0f096b6eed435b3db4a574ffba34a75a0abd502e9f8cdf370bbbd"}
Oct 31 21:59:15.952612 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:15.952546    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:59:16.109592 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:16.109552    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-workload-certs\") pod \"845e8f3e-1896-4417-bfe3-5d1638925a57\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") "
Oct 31 21:59:16.109797 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:16.109638    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-istio-data\") pod \"845e8f3e-1896-4417-bfe3-5d1638925a57\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") "
Oct 31 21:59:16.109797 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:16.109717    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/845e8f3e-1896-4417-bfe3-5d1638925a57-kube-api-access-lv2fv\" (UniqueName: \"kubernetes.io/projected/845e8f3e-1896-4417-bfe3-5d1638925a57-kube-api-access-lv2fv\") pod \"845e8f3e-1896-4417-bfe3-5d1638925a57\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") "
Oct 31 21:59:16.109797 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:16.109763    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/845e8f3e-1896-4417-bfe3-5d1638925a57-istio-token\" (UniqueName: \"kubernetes.io/projected/845e8f3e-1896-4417-bfe3-5d1638925a57-istio-token\") pod \"845e8f3e-1896-4417-bfe3-5d1638925a57\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") "
Oct 31 21:59:16.110043 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:16.109876    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-istio-envoy\") pod \"845e8f3e-1896-4417-bfe3-5d1638925a57\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") "
Oct 31 21:59:16.110043 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:16.109958    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/845e8f3e-1896-4417-bfe3-5d1638925a57-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/845e8f3e-1896-4417-bfe3-5d1638925a57-istio-podinfo\") pod \"845e8f3e-1896-4417-bfe3-5d1638925a57\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") "
Oct 31 21:59:16.110043 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:16.109915    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-workload-certs" pod "845e8f3e-1896-4417-bfe3-5d1638925a57" (UID: "845e8f3e-1896-4417-bfe3-5d1638925a57"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:59:16.110043 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:16.110004    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-workload-socket\") pod \"845e8f3e-1896-4417-bfe3-5d1638925a57\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") "
Oct 31 21:59:16.110043 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:16.110027    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-istio-data" pod "845e8f3e-1896-4417-bfe3-5d1638925a57" (UID: "845e8f3e-1896-4417-bfe3-5d1638925a57"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:59:16.110355 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:16.110109    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/845e8f3e-1896-4417-bfe3-5d1638925a57-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/845e8f3e-1896-4417-bfe3-5d1638925a57-istiod-ca-cert\") pod \"845e8f3e-1896-4417-bfe3-5d1638925a57\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") "
Oct 31 21:59:16.110355 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:16.110197    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-credential-socket\") pod \"845e8f3e-1896-4417-bfe3-5d1638925a57\" (UID: \"845e8f3e-1896-4417-bfe3-5d1638925a57\") "
Oct 31 21:59:16.110935 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:16.110908    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:59:16.110935 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:16.110935    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:59:16.111313 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:16.111197    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-credential-socket" pod "845e8f3e-1896-4417-bfe3-5d1638925a57" (UID: "845e8f3e-1896-4417-bfe3-5d1638925a57"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:59:16.111490 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:16.111448    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-workload-socket" pod "845e8f3e-1896-4417-bfe3-5d1638925a57" (UID: "845e8f3e-1896-4417-bfe3-5d1638925a57"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:59:16.111569 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:16.111551    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/845e8f3e-1896-4417-bfe3-5d1638925a57-istiod-ca-cert" pod "845e8f3e-1896-4417-bfe3-5d1638925a57" (UID: "845e8f3e-1896-4417-bfe3-5d1638925a57"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 21:59:16.112410 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:16.112386    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-istio-envoy" pod "845e8f3e-1896-4417-bfe3-5d1638925a57" (UID: "845e8f3e-1896-4417-bfe3-5d1638925a57"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 21:59:16.112555 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:16.112514    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/845e8f3e-1896-4417-bfe3-5d1638925a57-kube-api-access-lv2fv" pod "845e8f3e-1896-4417-bfe3-5d1638925a57" (UID: "845e8f3e-1896-4417-bfe3-5d1638925a57"). InnerVolumeSpecName "kube-api-access-lv2fv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:59:16.112622 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:16.112569    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/845e8f3e-1896-4417-bfe3-5d1638925a57-istio-podinfo" pod "845e8f3e-1896-4417-bfe3-5d1638925a57" (UID: "845e8f3e-1896-4417-bfe3-5d1638925a57"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 21:59:16.112683 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:16.112668    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/845e8f3e-1896-4417-bfe3-5d1638925a57-istio-token" pod "845e8f3e-1896-4417-bfe3-5d1638925a57" (UID: "845e8f3e-1896-4417-bfe3-5d1638925a57"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 21:59:16.212345 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:16.212262    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/845e8f3e-1896-4417-bfe3-5d1638925a57-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:59:16.212345 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:16.212296    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:59:16.212345 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:16.212306    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/845e8f3e-1896-4417-bfe3-5d1638925a57-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:59:16.212345 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:16.212314    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:59:16.212345 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:16.212325    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lv2fv\" (UniqueName: \"kubernetes.io/projected/845e8f3e-1896-4417-bfe3-5d1638925a57-kube-api-access-lv2fv\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:59:16.212345 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:16.212335    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/845e8f3e-1896-4417-bfe3-5d1638925a57-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:59:16.212345 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:16.212343    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/845e8f3e-1896-4417-bfe3-5d1638925a57-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 21:59:16.904391 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:16.904355    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"
Oct 31 21:59:16.904864 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:16.904387    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m" event={"ID":"845e8f3e-1896-4417-bfe3-5d1638925a57","Type":"ContainerDied","Data":"b590cd40663939f0a087cd66ef85d50c6610801590ad02ded1f317ec9b0102e6"}
Oct 31 21:59:16.904864 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:16.904458    4460 scope.go:122] "RemoveContainer" containerID="27c3d84586b0f096b6eed435b3db4a574ffba34a75a0abd502e9f8cdf370bbbd"
Oct 31 21:59:16.978782 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:16.978748    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"]
Oct 31 21:59:16.983764 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:16.983629    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-994336-openshift-default-5dd7866d79-zrd7m"]
Oct 31 21:59:17.038019 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:17.037983    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:18.226874 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:18.226844    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="845e8f3e-1896-4417-bfe3-5d1638925a57" path="/var/lib/kubelet/pods/845e8f3e-1896-4417-bfe3-5d1638925a57/volumes"
Oct 31 21:59:18.815394 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:18.815356    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:19.849236 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:19.849204    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a5affe-78c59c567-c6hdt"
Oct 31 21:59:20.446581 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:20.446544    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:21.454423 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:59:21.454383    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:59:22.131850 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:22.131807    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:23.807950 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:23.807905    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:25.522386 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:25.522347    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:27.224859 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:27.224821    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:27.943335 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:27.943295    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:28.061825 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:28.061787    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:29.137059 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:29.137023    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:30.907193 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:30.907150    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:32.536750 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:32.536718    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:34.064403 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:34.064357    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:34.170834 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:34.170787    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:34.278746 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:34.278610    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:35.929281 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:35.929244    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:37.563168 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:37.563119    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:39.276859 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:39.276813    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:39.963386 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:39.963280    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:40.078349 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:40.078305    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:40.585305 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:40.585267    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:41.030260 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:41.030224    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:42.648823 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:42.648776    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:44.391651 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:44.391618    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:44.516956 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:44.516901    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:46.152930 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:46.152890    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:47.019807 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:47.019758    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:47.143957 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:47.143905    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:47.254397 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:47.254352    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:47.880282 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:47.880245    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:48.916551 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:48.916512    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:49.544794 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:49.544757    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:51.314104 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:51.314070    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:51.454037 ip-10-0-136-157 kubenswrapper[4460]: E1031 21:59:51.454001    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 21:59:52.998156 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:52.998119    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:53.475703 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:53.475667    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:53.582820 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:53.582784    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:53.964769 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:53.964731    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:54.184397 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:54.184300    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:54.585650 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:54.585611    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 21:59:55.027220 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:55.027179    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:55.158021 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:55.157988    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:55.287471 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:55.287388    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs"]
Oct 31 21:59:55.288175 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:55.288156    4460 cpu_manager.go:414] "containerMap: removing container" podUID="845e8f3e-1896-4417-bfe3-5d1638925a57" containerName="istio-proxy"
Oct 31 21:59:55.288235 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:55.288176    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="845e8f3e-1896-4417-bfe3-5d1638925a57" containerName="istio-proxy"
Oct 31 21:59:55.288315 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:55.288305    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="845e8f3e-1896-4417-bfe3-5d1638925a57" containerName="istio-proxy"
Oct 31 21:59:55.292284 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:55.292264    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs"
Oct 31 21:59:55.310422 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:55.310394    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs"]
Oct 31 21:59:55.319022 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:55.318995    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-943963\"/\"e2e-cmp-legacy-d0dd68-provider-cred\""
Oct 31 21:59:55.319232 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:55.319160    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-943963\"/\"payload-processing-external-model-e2e--1e97e1d3-dockercfg-dh7dk\""
Oct 31 21:59:55.319326 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:55.319241    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-943963\"/\"kube-root-ca.crt\""
Oct 31 21:59:55.319399 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:55.319334    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-943963\"/\"openshift-service-ca.crt\""
Oct 31 21:59:55.324606 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:55.324580    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-943963\"/\"payload-processing-external-model-plugins-e2e-extmod-a-943963\""
Oct 31 21:59:55.324606 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:55.324587    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-943963\"/\"routing-overlay\""
Oct 31 21:59:55.516333 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:55.516223    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/231459dd-b722-4b40-ac48-a0d2271a3dcf-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs\" (UID: \"231459dd-b722-4b40-ac48-a0d2271a3dcf\") " pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs"
Oct 31 21:59:55.519223 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:55.519188    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/231459dd-b722-4b40-ac48-a0d2271a3dcf-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs\" (UID: \"231459dd-b722-4b40-ac48-a0d2271a3dcf\") " pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs"
Oct 31 21:59:55.519572 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:55.519541    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8rzd\" (UniqueName: \"kubernetes.io/projected/231459dd-b722-4b40-ac48-a0d2271a3dcf-kube-api-access-z8rzd\") pod \"payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs\" (UID: \"231459dd-b722-4b40-ac48-a0d2271a3dcf\") " pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs"
Oct 31 21:59:55.519754 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:55.519733    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/231459dd-b722-4b40-ac48-a0d2271a3dcf-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs\" (UID: \"231459dd-b722-4b40-ac48-a0d2271a3dcf\") " pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs"
Oct 31 21:59:55.621704 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:55.621611    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/231459dd-b722-4b40-ac48-a0d2271a3dcf-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs\" (UID: \"231459dd-b722-4b40-ac48-a0d2271a3dcf\") " pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs"
Oct 31 21:59:55.622469 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:55.622441    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/231459dd-b722-4b40-ac48-a0d2271a3dcf-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs\" (UID: \"231459dd-b722-4b40-ac48-a0d2271a3dcf\") " pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs"
Oct 31 21:59:55.637116 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:55.623513    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/231459dd-b722-4b40-ac48-a0d2271a3dcf-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs\" (UID: \"231459dd-b722-4b40-ac48-a0d2271a3dcf\") " pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs"
Oct 31 21:59:55.645378 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:55.645347    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8rzd\" (UniqueName: \"kubernetes.io/projected/231459dd-b722-4b40-ac48-a0d2271a3dcf-kube-api-access-z8rzd\") pod \"payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs\" (UID: \"231459dd-b722-4b40-ac48-a0d2271a3dcf\") " pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs"
Oct 31 21:59:55.647081 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:55.647055    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/231459dd-b722-4b40-ac48-a0d2271a3dcf-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs\" (UID: \"231459dd-b722-4b40-ac48-a0d2271a3dcf\") " pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs"
Oct 31 21:59:55.647268 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:55.645804    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/231459dd-b722-4b40-ac48-a0d2271a3dcf-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs\" (UID: \"231459dd-b722-4b40-ac48-a0d2271a3dcf\") " pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs"
Oct 31 21:59:55.668144 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:55.655752    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/231459dd-b722-4b40-ac48-a0d2271a3dcf-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs\" (UID: \"231459dd-b722-4b40-ac48-a0d2271a3dcf\") " pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs"
Oct 31 21:59:55.687052 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:55.679897    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8rzd\" (UniqueName: \"kubernetes.io/projected/231459dd-b722-4b40-ac48-a0d2271a3dcf-kube-api-access-z8rzd\") pod \"payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs\" (UID: \"231459dd-b722-4b40-ac48-a0d2271a3dcf\") " pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs"
Oct 31 21:59:55.909412 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:55.909321    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs"
Oct 31 21:59:56.037110 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:56.037068    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:56.161320 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:56.161221    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:56.227720 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:56.227687    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs"]
Oct 31 21:59:56.229321 ip-10-0-136-157 kubenswrapper[4460]: W1031 21:59:56.229294    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod231459dd_b722_4b40_ac48_a0d2271a3dcf.slice/crio-dd43368ec17f3ab07a041093cc055bb537b87f38ae0b49c34cc3e7268e17728a WatchSource:0}: Error finding container dd43368ec17f3ab07a041093cc055bb537b87f38ae0b49c34cc3e7268e17728a: Status 404 returned error can't find the container with id dd43368ec17f3ab07a041093cc055bb537b87f38ae0b49c34cc3e7268e17728a
Oct 31 21:59:56.255893 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:56.255856    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs" event={"ID":"231459dd-b722-4b40-ac48-a0d2271a3dcf","Type":"ContainerStarted","Data":"dd43368ec17f3ab07a041093cc055bb537b87f38ae0b49c34cc3e7268e17728a"}
Oct 31 21:59:56.790223 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:56.790174    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:57.265568 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:57.265528    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs" event={"ID":"231459dd-b722-4b40-ac48-a0d2271a3dcf","Type":"ContainerStarted","Data":"c45048664b7a21ff780b2f99ae21eda33407f7347367dc51389ec8eb69cbeec7"}
Oct 31 21:59:57.265744 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:57.265724    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs"
Oct 31 21:59:57.316403 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:57.316333    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs" podStartSLOduration=2.316312791 podStartE2EDuration="2.316312791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 21:59:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 21:59:57.310374959 +0000 UTC m=+2637.226684389" watchObservedRunningTime="2026-10-31 21:59:57.316312791 +0000 UTC m=+2637.232622222"
Oct 31 21:59:58.426958 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:58.426906    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:59.234105 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:59.234073    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 21:59:59.356585 ip-10-0-136-157 kubenswrapper[4460]: I1031 21:59:59.356547    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:00.147456 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.147416    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq"]
Oct 31 22:00:00.152379 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.152352    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk"]
Oct 31 22:00:00.152557 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.152488    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq"
Oct 31 22:00:00.157878 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.157850    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk"
Oct 31 22:00:00.174178 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.173799    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j"]
Oct 31 22:00:00.183453 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.180292    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9hwvp\""
Oct 31 22:00:00.183453 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.180298    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk"]
Oct 31 22:00:00.183453 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.180384    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j"
Oct 31 22:00:00.213507 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.213472    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq"]
Oct 31 22:00:00.213705 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.213629    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk"
Oct 31 22:00:00.266018 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.256654    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gt9rp\" (UniqueName: \"kubernetes.io/projected/31d27730-27b7-4bfa-ac38-5a4a3cc13487-kube-api-access-gt9rp\") pod \"maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk\" (UID: \"31d27730-27b7-4bfa-ac38-5a4a3cc13487\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk"
Oct 31 22:00:00.267001 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.266283    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/31d27730-27b7-4bfa-ac38-5a4a3cc13487-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk\" (UID: \"31d27730-27b7-4bfa-ac38-5a4a3cc13487\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk"
Oct 31 22:00:00.267492 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.267454    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/229deecb-4066-4ef7-bcf0-70bd5c15df45-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq\" (UID: \"229deecb-4066-4ef7-bcf0-70bd5c15df45\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq"
Oct 31 22:00:00.267992 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.267694    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vp2f5\" (UniqueName: \"kubernetes.io/projected/229deecb-4066-4ef7-bcf0-70bd5c15df45-kube-api-access-vp2f5\") pod \"maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq\" (UID: \"229deecb-4066-4ef7-bcf0-70bd5c15df45\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq"
Oct 31 22:00:00.284110 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.283261    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk"]
Oct 31 22:00:00.284110 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.283326    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n"]
Oct 31 22:00:00.295469 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.294957    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j"]
Oct 31 22:00:00.295469 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.295011    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk"]
Oct 31 22:00:00.295469 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.295124    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n"
Oct 31 22:00:00.295469 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.295157    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r"]
Oct 31 22:00:00.299412 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.298053    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:00.306110 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.305463    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f"]
Oct 31 22:00:00.306460 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.306436    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r"
Oct 31 22:00:00.314244 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.314220    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n"]
Oct 31 22:00:00.314417 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.314267    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w"]
Oct 31 22:00:00.318204 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.318180    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f"
Oct 31 22:00:00.321069 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.321044    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w"]
Oct 31 22:00:00.322118 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.322094    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w"
Oct 31 22:00:00.374020 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.371599    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f"]
Oct 31 22:00:00.378815 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.378787    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23b20e87-150e-48d7-9da9-d6c81de609d7-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk\" (UID: \"23b20e87-150e-48d7-9da9-d6c81de609d7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk"
Oct 31 22:00:00.379109 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.379089    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/229deecb-4066-4ef7-bcf0-70bd5c15df45-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq\" (UID: \"229deecb-4066-4ef7-bcf0-70bd5c15df45\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq"
Oct 31 22:00:00.379381 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.379354    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vp2f5\" (UniqueName: \"kubernetes.io/projected/229deecb-4066-4ef7-bcf0-70bd5c15df45-kube-api-access-vp2f5\") pod \"maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq\" (UID: \"229deecb-4066-4ef7-bcf0-70bd5c15df45\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq"
Oct 31 22:00:00.379609 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.379526    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj"]
Oct 31 22:00:00.381264 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.381238    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w"
Oct 31 22:00:00.381368 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.380735    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gt9rp\" (UniqueName: \"kubernetes.io/projected/31d27730-27b7-4bfa-ac38-5a4a3cc13487-kube-api-access-gt9rp\") pod \"maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk\" (UID: \"31d27730-27b7-4bfa-ac38-5a4a3cc13487\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk"
Oct 31 22:00:00.381460 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.381447    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/31d27730-27b7-4bfa-ac38-5a4a3cc13487-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk\" (UID: \"31d27730-27b7-4bfa-ac38-5a4a3cc13487\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk"
Oct 31 22:00:00.381546 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.381533    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghw5b\" (UniqueName: \"kubernetes.io/projected/0a11ccff-84e3-4179-9f19-f07b86783bd3-kube-api-access-ghw5b\") pod \"maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j\" (UID: \"0a11ccff-84e3-4179-9f19-f07b86783bd3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j"
Oct 31 22:00:00.381623 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.381613    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56cqw\" (UniqueName: \"kubernetes.io/projected/23b20e87-150e-48d7-9da9-d6c81de609d7-kube-api-access-56cqw\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk\" (UID: \"23b20e87-150e-48d7-9da9-d6c81de609d7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk"
Oct 31 22:00:00.381707 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.381698    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a11ccff-84e3-4179-9f19-f07b86783bd3-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j\" (UID: \"0a11ccff-84e3-4179-9f19-f07b86783bd3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j"
Oct 31 22:00:00.381874 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.381862    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/229deecb-4066-4ef7-bcf0-70bd5c15df45-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq\" (UID: \"229deecb-4066-4ef7-bcf0-70bd5c15df45\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq"
Oct 31 22:00:00.382168 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.382149    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/31d27730-27b7-4bfa-ac38-5a4a3cc13487-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk\" (UID: \"31d27730-27b7-4bfa-ac38-5a4a3cc13487\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk"
Oct 31 22:00:00.395283 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.395045    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r"]
Oct 31 22:00:00.395283 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.395161    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj"
Oct 31 22:00:00.406084 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.401220    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w"]
Oct 31 22:00:00.406215 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.406130    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gt9rp\" (UniqueName: \"kubernetes.io/projected/31d27730-27b7-4bfa-ac38-5a4a3cc13487-kube-api-access-gt9rp\") pod \"maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk\" (UID: \"31d27730-27b7-4bfa-ac38-5a4a3cc13487\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk"
Oct 31 22:00:00.406802 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.406781    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w"]
Oct 31 22:00:00.408548 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.408521    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vp2f5\" (UniqueName: \"kubernetes.io/projected/229deecb-4066-4ef7-bcf0-70bd5c15df45-kube-api-access-vp2f5\") pod \"maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq\" (UID: \"229deecb-4066-4ef7-bcf0-70bd5c15df45\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq"
Oct 31 22:00:00.414635 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.414607    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj"]
Oct 31 22:00:00.482654 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.482599    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb824101-1f12-403b-bf06-1051f6f25d17-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n\" (UID: \"fb824101-1f12-403b-bf06-1051f6f25d17\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n"
Oct 31 22:00:00.482834 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.482663    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e081a034-3ce7-4af9-ba84-d44f463ae8fe-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj\" (UID: \"e081a034-3ce7-4af9-ba84-d44f463ae8fe\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj"
Oct 31 22:00:00.482834 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.482723    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbnc8\" (UniqueName: \"kubernetes.io/projected/fb824101-1f12-403b-bf06-1051f6f25d17-kube-api-access-bbnc8\") pod \"maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n\" (UID: \"fb824101-1f12-403b-bf06-1051f6f25d17\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n"
Oct 31 22:00:00.482834 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.482825    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smjk8\" (UniqueName: \"kubernetes.io/projected/0864e283-2018-4c62-abeb-809f9dd12ce4-kube-api-access-smjk8\") pod \"maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w\" (UID: \"0864e283-2018-4c62-abeb-809f9dd12ce4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w"
Oct 31 22:00:00.482992 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.482887    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxqbm\" (UniqueName: \"kubernetes.io/projected/63aa3078-082d-400e-abfb-9985668831fc-kube-api-access-wxqbm\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w\" (UID: \"63aa3078-082d-400e-abfb-9985668831fc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w"
Oct 31 22:00:00.482992 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.482917    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cv6b2\" (UniqueName: \"kubernetes.io/projected/e081a034-3ce7-4af9-ba84-d44f463ae8fe-kube-api-access-cv6b2\") pod \"maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj\" (UID: \"e081a034-3ce7-4af9-ba84-d44f463ae8fe\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj"
Oct 31 22:00:00.482992 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.482979    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gntm\" (UniqueName: \"kubernetes.io/projected/093b9d9a-ec69-4552-a889-6f3bccda27cc-kube-api-access-9gntm\") pod \"maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r\" (UID: \"093b9d9a-ec69-4552-a889-6f3bccda27cc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r"
Oct 31 22:00:00.483105 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.483034    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wt5r7\" (UniqueName: \"kubernetes.io/projected/76255444-0212-46b9-9a6d-571bc0ba8f28-kube-api-access-wt5r7\") pod \"maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f\" (UID: \"76255444-0212-46b9-9a6d-571bc0ba8f28\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f"
Oct 31 22:00:00.483147 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.483109    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghw5b\" (UniqueName: \"kubernetes.io/projected/0a11ccff-84e3-4179-9f19-f07b86783bd3-kube-api-access-ghw5b\") pod \"maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j\" (UID: \"0a11ccff-84e3-4179-9f19-f07b86783bd3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j"
Oct 31 22:00:00.483188 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.483157    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/76255444-0212-46b9-9a6d-571bc0ba8f28-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f\" (UID: \"76255444-0212-46b9-9a6d-571bc0ba8f28\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f"
Oct 31 22:00:00.483188 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.483181    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/093b9d9a-ec69-4552-a889-6f3bccda27cc-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r\" (UID: \"093b9d9a-ec69-4552-a889-6f3bccda27cc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r"
Oct 31 22:00:00.483259 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.483207    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56cqw\" (UniqueName: \"kubernetes.io/projected/23b20e87-150e-48d7-9da9-d6c81de609d7-kube-api-access-56cqw\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk\" (UID: \"23b20e87-150e-48d7-9da9-d6c81de609d7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk"
Oct 31 22:00:00.483301 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.483285    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a11ccff-84e3-4179-9f19-f07b86783bd3-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j\" (UID: \"0a11ccff-84e3-4179-9f19-f07b86783bd3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j"
Oct 31 22:00:00.483338 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.483311    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0864e283-2018-4c62-abeb-809f9dd12ce4-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w\" (UID: \"0864e283-2018-4c62-abeb-809f9dd12ce4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w"
Oct 31 22:00:00.483338 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.483332    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63aa3078-082d-400e-abfb-9985668831fc-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w\" (UID: \"63aa3078-082d-400e-abfb-9985668831fc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w"
Oct 31 22:00:00.483427 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.483373    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23b20e87-150e-48d7-9da9-d6c81de609d7-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk\" (UID: \"23b20e87-150e-48d7-9da9-d6c81de609d7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk"
Oct 31 22:00:00.484061 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.484042    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23b20e87-150e-48d7-9da9-d6c81de609d7-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk\" (UID: \"23b20e87-150e-48d7-9da9-d6c81de609d7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk"
Oct 31 22:00:00.484130 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.484098    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a11ccff-84e3-4179-9f19-f07b86783bd3-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j\" (UID: \"0a11ccff-84e3-4179-9f19-f07b86783bd3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j"
Oct 31 22:00:00.496332 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.496301    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9hwvp\""
Oct 31 22:00:00.501492 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.501466    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq"
Oct 31 22:00:00.510665 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.510597    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghw5b\" (UniqueName: \"kubernetes.io/projected/0a11ccff-84e3-4179-9f19-f07b86783bd3-kube-api-access-ghw5b\") pod \"maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j\" (UID: \"0a11ccff-84e3-4179-9f19-f07b86783bd3\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j"
Oct 31 22:00:00.511819 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.511786    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56cqw\" (UniqueName: \"kubernetes.io/projected/23b20e87-150e-48d7-9da9-d6c81de609d7-kube-api-access-56cqw\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk\" (UID: \"23b20e87-150e-48d7-9da9-d6c81de609d7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk"
Oct 31 22:00:00.529554 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.529524    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk"
Oct 31 22:00:00.601245 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.584634    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/76255444-0212-46b9-9a6d-571bc0ba8f28-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f\" (UID: \"76255444-0212-46b9-9a6d-571bc0ba8f28\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f"
Oct 31 22:00:00.601245 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.584683    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/093b9d9a-ec69-4552-a889-6f3bccda27cc-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r\" (UID: \"093b9d9a-ec69-4552-a889-6f3bccda27cc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r"
Oct 31 22:00:00.601245 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.585758    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/76255444-0212-46b9-9a6d-571bc0ba8f28-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f\" (UID: \"76255444-0212-46b9-9a6d-571bc0ba8f28\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f"
Oct 31 22:00:00.601245 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.594708    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j"
Oct 31 22:00:00.601245 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.595678    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0864e283-2018-4c62-abeb-809f9dd12ce4-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w\" (UID: \"0864e283-2018-4c62-abeb-809f9dd12ce4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w"
Oct 31 22:00:00.601245 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.584735    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0864e283-2018-4c62-abeb-809f9dd12ce4-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w\" (UID: \"0864e283-2018-4c62-abeb-809f9dd12ce4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w"
Oct 31 22:00:00.601245 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.598080    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63aa3078-082d-400e-abfb-9985668831fc-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w\" (UID: \"63aa3078-082d-400e-abfb-9985668831fc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w"
Oct 31 22:00:00.601245 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.598209    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb824101-1f12-403b-bf06-1051f6f25d17-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n\" (UID: \"fb824101-1f12-403b-bf06-1051f6f25d17\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n"
Oct 31 22:00:00.601245 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.598245    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e081a034-3ce7-4af9-ba84-d44f463ae8fe-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj\" (UID: \"e081a034-3ce7-4af9-ba84-d44f463ae8fe\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj"
Oct 31 22:00:00.601245 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.598275    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbnc8\" (UniqueName: \"kubernetes.io/projected/fb824101-1f12-403b-bf06-1051f6f25d17-kube-api-access-bbnc8\") pod \"maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n\" (UID: \"fb824101-1f12-403b-bf06-1051f6f25d17\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n"
Oct 31 22:00:00.601245 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.598422    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smjk8\" (UniqueName: \"kubernetes.io/projected/0864e283-2018-4c62-abeb-809f9dd12ce4-kube-api-access-smjk8\") pod \"maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w\" (UID: \"0864e283-2018-4c62-abeb-809f9dd12ce4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w"
Oct 31 22:00:00.601245 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.598511    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxqbm\" (UniqueName: \"kubernetes.io/projected/63aa3078-082d-400e-abfb-9985668831fc-kube-api-access-wxqbm\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w\" (UID: \"63aa3078-082d-400e-abfb-9985668831fc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w"
Oct 31 22:00:00.601245 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.598567    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv6b2\" (UniqueName: \"kubernetes.io/projected/e081a034-3ce7-4af9-ba84-d44f463ae8fe-kube-api-access-cv6b2\") pod \"maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj\" (UID: \"e081a034-3ce7-4af9-ba84-d44f463ae8fe\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj"
Oct 31 22:00:00.601245 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.598617    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gntm\" (UniqueName: \"kubernetes.io/projected/093b9d9a-ec69-4552-a889-6f3bccda27cc-kube-api-access-9gntm\") pod \"maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r\" (UID: \"093b9d9a-ec69-4552-a889-6f3bccda27cc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r"
Oct 31 22:00:00.601245 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.598686    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt5r7\" (UniqueName: \"kubernetes.io/projected/76255444-0212-46b9-9a6d-571bc0ba8f28-kube-api-access-wt5r7\") pod \"maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f\" (UID: \"76255444-0212-46b9-9a6d-571bc0ba8f28\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f"
Oct 31 22:00:00.601245 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.599855    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63aa3078-082d-400e-abfb-9985668831fc-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w\" (UID: \"63aa3078-082d-400e-abfb-9985668831fc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w"
Oct 31 22:00:00.601245 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.600744    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e081a034-3ce7-4af9-ba84-d44f463ae8fe-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj\" (UID: \"e081a034-3ce7-4af9-ba84-d44f463ae8fe\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj"
Oct 31 22:00:00.602727 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.602231    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk"
Oct 31 22:00:00.602727 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.602682    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb824101-1f12-403b-bf06-1051f6f25d17-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n\" (UID: \"fb824101-1f12-403b-bf06-1051f6f25d17\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n"
Oct 31 22:00:00.626961 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.603725    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/093b9d9a-ec69-4552-a889-6f3bccda27cc-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r\" (UID: \"093b9d9a-ec69-4552-a889-6f3bccda27cc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r"
Oct 31 22:00:00.640658 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.640544    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wt5r7\" (UniqueName: \"kubernetes.io/projected/76255444-0212-46b9-9a6d-571bc0ba8f28-kube-api-access-wt5r7\") pod \"maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f\" (UID: \"76255444-0212-46b9-9a6d-571bc0ba8f28\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f"
Oct 31 22:00:00.640658 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.640621    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cv6b2\" (UniqueName: \"kubernetes.io/projected/e081a034-3ce7-4af9-ba84-d44f463ae8fe-kube-api-access-cv6b2\") pod \"maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj\" (UID: \"e081a034-3ce7-4af9-ba84-d44f463ae8fe\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj"
Oct 31 22:00:00.642031 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.641975    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gntm\" (UniqueName: \"kubernetes.io/projected/093b9d9a-ec69-4552-a889-6f3bccda27cc-kube-api-access-9gntm\") pod \"maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r\" (UID: \"093b9d9a-ec69-4552-a889-6f3bccda27cc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r"
Oct 31 22:00:00.642811 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.642768    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-smjk8\" (UniqueName: \"kubernetes.io/projected/0864e283-2018-4c62-abeb-809f9dd12ce4-kube-api-access-smjk8\") pod \"maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w\" (UID: \"0864e283-2018-4c62-abeb-809f9dd12ce4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w"
Oct 31 22:00:00.643569 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.643525    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbnc8\" (UniqueName: \"kubernetes.io/projected/fb824101-1f12-403b-bf06-1051f6f25d17-kube-api-access-bbnc8\") pod \"maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n\" (UID: \"fb824101-1f12-403b-bf06-1051f6f25d17\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n"
Oct 31 22:00:00.644689 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.644655    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxqbm\" (UniqueName: \"kubernetes.io/projected/63aa3078-082d-400e-abfb-9985668831fc-kube-api-access-wxqbm\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w\" (UID: \"63aa3078-082d-400e-abfb-9985668831fc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w"
Oct 31 22:00:00.683321 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.682847    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n"
Oct 31 22:00:00.701046 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.701006    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f"
Oct 31 22:00:00.701213 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.701155    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r"
Oct 31 22:00:00.742782 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.742749    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w"
Oct 31 22:00:00.750275 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.750243    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w"
Oct 31 22:00:00.758162 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.758131    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj"
Oct 31 22:00:00.834958 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.831799    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg"]
Oct 31 22:00:00.841382 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.841354    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg"
Oct 31 22:00:00.842226 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.841522    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq"]
Oct 31 22:00:00.864437 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.858658    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg"]
Oct 31 22:00:00.864437 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.861143    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-fac043\"/\"agc-cred-b06b04d8d7ad\""
Oct 31 22:00:00.889119 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.889084    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk"]
Oct 31 22:00:00.931014 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.930491    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/7c2a2f28-904a-4aef-96dd-3165e1f391db-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg\" (UID: \"7c2a2f28-904a-4aef-96dd-3165e1f391db\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg"
Oct 31 22:00:00.931014 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.930636    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7c2a2f28-904a-4aef-96dd-3165e1f391db-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg\" (UID: \"7c2a2f28-904a-4aef-96dd-3165e1f391db\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg"
Oct 31 22:00:00.931014 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.930688    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c2a2f28-904a-4aef-96dd-3165e1f391db-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg\" (UID: \"7c2a2f28-904a-4aef-96dd-3165e1f391db\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg"
Oct 31 22:00:00.931014 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.930740    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k99px\" (UniqueName: \"kubernetes.io/projected/7c2a2f28-904a-4aef-96dd-3165e1f391db-kube-api-access-k99px\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg\" (UID: \"7c2a2f28-904a-4aef-96dd-3165e1f391db\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg"
Oct 31 22:00:00.979988 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:00.976217    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:01.032906 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:01.032874    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/7c2a2f28-904a-4aef-96dd-3165e1f391db-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg\" (UID: \"7c2a2f28-904a-4aef-96dd-3165e1f391db\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg"
Oct 31 22:00:01.034001 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:01.033978    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7c2a2f28-904a-4aef-96dd-3165e1f391db-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg\" (UID: \"7c2a2f28-904a-4aef-96dd-3165e1f391db\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg"
Oct 31 22:00:01.034138 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:01.034027    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c2a2f28-904a-4aef-96dd-3165e1f391db-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg\" (UID: \"7c2a2f28-904a-4aef-96dd-3165e1f391db\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg"
Oct 31 22:00:01.034138 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:01.034067    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k99px\" (UniqueName: \"kubernetes.io/projected/7c2a2f28-904a-4aef-96dd-3165e1f391db-kube-api-access-k99px\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg\" (UID: \"7c2a2f28-904a-4aef-96dd-3165e1f391db\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg"
Oct 31 22:00:01.035017 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:01.034997    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7c2a2f28-904a-4aef-96dd-3165e1f391db-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg\" (UID: \"7c2a2f28-904a-4aef-96dd-3165e1f391db\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg"
Oct 31 22:00:01.035569 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:01.035550    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c2a2f28-904a-4aef-96dd-3165e1f391db-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg\" (UID: \"7c2a2f28-904a-4aef-96dd-3165e1f391db\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg"
Oct 31 22:00:01.091809 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:01.091769    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:01.122128 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:01.122098    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/7c2a2f28-904a-4aef-96dd-3165e1f391db-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg\" (UID: \"7c2a2f28-904a-4aef-96dd-3165e1f391db\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg"
Oct 31 22:00:01.122309 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:01.122156    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k99px\" (UniqueName: \"kubernetes.io/projected/7c2a2f28-904a-4aef-96dd-3165e1f391db-kube-api-access-k99px\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg\" (UID: \"7c2a2f28-904a-4aef-96dd-3165e1f391db\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg"
Oct 31 22:00:01.204518 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:01.197988    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg"
Oct 31 22:00:01.260217 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:01.257364    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j"]
Oct 31 22:00:01.270252 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:01.270214    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk"]
Oct 31 22:00:01.271366 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:01.270740    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w"]
Oct 31 22:00:01.309482 ip-10-0-136-157 kubenswrapper[4460]: W1031 22:00:01.309440    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0864e283_2018_4c62_abeb_809f9dd12ce4.slice/crio-3ae701caacbdbc9274eeb9eac5c8530c87701018d2f65b2f14f552a3b77e6265 WatchSource:0}: Error finding container 3ae701caacbdbc9274eeb9eac5c8530c87701018d2f65b2f14f552a3b77e6265: Status 404 returned error can't find the container with id 3ae701caacbdbc9274eeb9eac5c8530c87701018d2f65b2f14f552a3b77e6265
Oct 31 22:00:01.315032 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:01.313384    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n"]
Oct 31 22:00:01.395197 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:01.319687    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj"]
Oct 31 22:00:01.395197 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:01.335973    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w"]
Oct 31 22:00:01.516498 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:01.516464    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r"]
Oct 31 22:00:01.516770 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:01.516750    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f"]
Oct 31 22:00:01.535351 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:01.535298    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk" event={"ID":"31d27730-27b7-4bfa-ac38-5a4a3cc13487","Type":"ContainerStarted","Data":"c48b80364022daf20c49d5127a0427f5ec68c3f5c4ee41139e01bd8a88ceae16"}
Oct 31 22:00:01.541288 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:01.541250    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq" event={"ID":"229deecb-4066-4ef7-bcf0-70bd5c15df45","Type":"ContainerStarted","Data":"d5928258fd07d86b318ebef3cf596a1859a64a6440283296611dbc97096dcc6e"}
Oct 31 22:00:01.836365 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:01.836333    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg"]
Oct 31 22:00:01.845885 ip-10-0-136-157 kubenswrapper[4460]: W1031 22:00:01.845553    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c2a2f28_904a_4aef_96dd_3165e1f391db.slice/crio-259ccfe6232c6a249ea3baa2e8ad6ce735f8123901fd2e3ac970c35bbd817669 WatchSource:0}: Error finding container 259ccfe6232c6a249ea3baa2e8ad6ce735f8123901fd2e3ac970c35bbd817669: Status 404 returned error can't find the container with id 259ccfe6232c6a249ea3baa2e8ad6ce735f8123901fd2e3ac970c35bbd817669
Oct 31 22:00:01.894733 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:01.894699    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg"]
Oct 31 22:00:01.931929 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:01.931898    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj"]
Oct 31 22:00:01.937296 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:01.937270    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj"
Oct 31 22:00:01.957920 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:01.955024    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-fac043\"/\"agc-cred-5e1f9daadd79\""
Oct 31 22:00:01.957920 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:01.955364    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj"]
Oct 31 22:00:02.077427 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.077387    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69ea00be-260e-490a-bafd-e629d23f57cb-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj\" (UID: \"69ea00be-260e-490a-bafd-e629d23f57cb\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj"
Oct 31 22:00:02.077427 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.077429    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/69ea00be-260e-490a-bafd-e629d23f57cb-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj\" (UID: \"69ea00be-260e-490a-bafd-e629d23f57cb\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj"
Oct 31 22:00:02.077725 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.077697    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ktzt\" (UniqueName: \"kubernetes.io/projected/69ea00be-260e-490a-bafd-e629d23f57cb-kube-api-access-8ktzt\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj\" (UID: \"69ea00be-260e-490a-bafd-e629d23f57cb\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj"
Oct 31 22:00:02.077768 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.077748    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/69ea00be-260e-490a-bafd-e629d23f57cb-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj\" (UID: \"69ea00be-260e-490a-bafd-e629d23f57cb\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj"
Oct 31 22:00:02.179274 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.179242    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ktzt\" (UniqueName: \"kubernetes.io/projected/69ea00be-260e-490a-bafd-e629d23f57cb-kube-api-access-8ktzt\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj\" (UID: \"69ea00be-260e-490a-bafd-e629d23f57cb\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj"
Oct 31 22:00:02.179439 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.179290    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/69ea00be-260e-490a-bafd-e629d23f57cb-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj\" (UID: \"69ea00be-260e-490a-bafd-e629d23f57cb\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj"
Oct 31 22:00:02.179439 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.179428    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69ea00be-260e-490a-bafd-e629d23f57cb-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj\" (UID: \"69ea00be-260e-490a-bafd-e629d23f57cb\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj"
Oct 31 22:00:02.179546 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.179454    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/69ea00be-260e-490a-bafd-e629d23f57cb-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj\" (UID: \"69ea00be-260e-490a-bafd-e629d23f57cb\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj"
Oct 31 22:00:02.180224 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.180194    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69ea00be-260e-490a-bafd-e629d23f57cb-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj\" (UID: \"69ea00be-260e-490a-bafd-e629d23f57cb\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj"
Oct 31 22:00:02.180428 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.180403    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/69ea00be-260e-490a-bafd-e629d23f57cb-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj\" (UID: \"69ea00be-260e-490a-bafd-e629d23f57cb\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj"
Oct 31 22:00:02.225896 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.225864    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/69ea00be-260e-490a-bafd-e629d23f57cb-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj\" (UID: \"69ea00be-260e-490a-bafd-e629d23f57cb\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj"
Oct 31 22:00:02.227209 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.227155    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ktzt\" (UniqueName: \"kubernetes.io/projected/69ea00be-260e-490a-bafd-e629d23f57cb-kube-api-access-8ktzt\") pod \"payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj\" (UID: \"69ea00be-260e-490a-bafd-e629d23f57cb\") " pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj"
Oct 31 22:00:02.237473 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.237434    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:02.311298 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.305607    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj"
Oct 31 22:00:02.550955 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.550902    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg" event={"ID":"7c2a2f28-904a-4aef-96dd-3165e1f391db","Type":"ContainerStarted","Data":"6984e616eb3156e04d937be254dd97ea84a8db0efc0dfc74bb0c4aaba359d454"}
Oct 31 22:00:02.551178 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.550984    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg" event={"ID":"7c2a2f28-904a-4aef-96dd-3165e1f391db","Type":"ContainerStarted","Data":"259ccfe6232c6a249ea3baa2e8ad6ce735f8123901fd2e3ac970c35bbd817669"}
Oct 31 22:00:02.551178 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.551116    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg"
Oct 31 22:00:02.552614 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.552579    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r" event={"ID":"093b9d9a-ec69-4552-a889-6f3bccda27cc","Type":"ContainerStarted","Data":"539d96f110256cf72bdb80ab7dfac478f9d21f50f999d9f11f63866b5c108ab6"}
Oct 31 22:00:02.552614 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.552617    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r" event={"ID":"093b9d9a-ec69-4552-a889-6f3bccda27cc","Type":"ContainerStarted","Data":"dbbdfcc9c466c875778d9ac469127cd59eb65520a96f8385bbc5237beb6039ec"}
Oct 31 22:00:02.554244 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.554217    4460 generic.go:356] "Generic (PLEG): container finished" podID="31d27730-27b7-4bfa-ac38-5a4a3cc13487" containerID="50ba29ebdf8ffc0b381a2efdd77dc226c4a27802344ccc9ef57aa865406c890e" exitCode=60
Oct 31 22:00:02.554359 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.554299    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk" event={"ID":"31d27730-27b7-4bfa-ac38-5a4a3cc13487","Type":"ContainerDied","Data":"50ba29ebdf8ffc0b381a2efdd77dc226c4a27802344ccc9ef57aa865406c890e"}
Oct 31 22:00:02.554599 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.554582    4460 scope.go:122] "RemoveContainer" containerID="50ba29ebdf8ffc0b381a2efdd77dc226c4a27802344ccc9ef57aa865406c890e"
Oct 31 22:00:02.555971 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.555947    4460 generic.go:356] "Generic (PLEG): container finished" podID="229deecb-4066-4ef7-bcf0-70bd5c15df45" containerID="183fd1fd41664c6467682918b9d6c98495cdff27fb79509ecdb5a24e8591934c" exitCode=60
Oct 31 22:00:02.556054 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.556020    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq" event={"ID":"229deecb-4066-4ef7-bcf0-70bd5c15df45","Type":"ContainerDied","Data":"183fd1fd41664c6467682918b9d6c98495cdff27fb79509ecdb5a24e8591934c"}
Oct 31 22:00:02.556272 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.556257    4460 scope.go:122] "RemoveContainer" containerID="183fd1fd41664c6467682918b9d6c98495cdff27fb79509ecdb5a24e8591934c"
Oct 31 22:00:02.558033 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.558010    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w" event={"ID":"63aa3078-082d-400e-abfb-9985668831fc","Type":"ContainerStarted","Data":"31306d2ac046511759a5fa2eb07103a04cc93cf72f613521c565f512b6b68a53"}
Oct 31 22:00:02.558136 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.558057    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w" event={"ID":"63aa3078-082d-400e-abfb-9985668831fc","Type":"ContainerStarted","Data":"f1ae326067255a89263e3d75cba17141ed94db7a2aeda6764457a3bfee63e55b"}
Oct 31 22:00:02.559669 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.559607    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w" event={"ID":"0864e283-2018-4c62-abeb-809f9dd12ce4","Type":"ContainerStarted","Data":"81a70702a7a2d04b6ae469085e9e5c4f2252661d14160b937b80337ad0ddbd32"}
Oct 31 22:00:02.559669 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.559638    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w" event={"ID":"0864e283-2018-4c62-abeb-809f9dd12ce4","Type":"ContainerStarted","Data":"3ae701caacbdbc9274eeb9eac5c8530c87701018d2f65b2f14f552a3b77e6265"}
Oct 31 22:00:02.561758 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.561733    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk" event={"ID":"23b20e87-150e-48d7-9da9-d6c81de609d7","Type":"ContainerStarted","Data":"0a26bcfd162255b4f9059392378c582608184246534700408974c77e748178c3"}
Oct 31 22:00:02.561861 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.561780    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk" event={"ID":"23b20e87-150e-48d7-9da9-d6c81de609d7","Type":"ContainerStarted","Data":"7051a729dc682b981606cfaa2202815bb8d03658889d6bfcbb4075f3c3954206"}
Oct 31 22:00:02.563257 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.563235    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f" event={"ID":"76255444-0212-46b9-9a6d-571bc0ba8f28","Type":"ContainerStarted","Data":"d960761c0c7638d9d0a06aa39332bcf031bdfe7929135921352ff9a7259cf09d"}
Oct 31 22:00:02.563355 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.563278    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f" event={"ID":"76255444-0212-46b9-9a6d-571bc0ba8f28","Type":"ContainerStarted","Data":"8e737645fe4f795f64652f49fdee7ac17608e98829a15c93cee1abc534662d07"}
Oct 31 22:00:02.564699 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.564676    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n" event={"ID":"fb824101-1f12-403b-bf06-1051f6f25d17","Type":"ContainerStarted","Data":"fa5878739604c3e1a008c0c4d572c93605209d81494c0b964ae937a3b7007767"}
Oct 31 22:00:02.564807 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.564722    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n" event={"ID":"fb824101-1f12-403b-bf06-1051f6f25d17","Type":"ContainerStarted","Data":"570728dceb250031f2c0d7a77b7fe37791a5efae709eaf6eec449a561ccd267a"}
Oct 31 22:00:02.566274 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.566243    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj" event={"ID":"e081a034-3ce7-4af9-ba84-d44f463ae8fe","Type":"ContainerStarted","Data":"e10a1d6e9264c4d9922a5612c82100b6ef0d3c0a5b7eed4985ea68424e4c52f3"}
Oct 31 22:00:02.566369 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.566286    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj" event={"ID":"e081a034-3ce7-4af9-ba84-d44f463ae8fe","Type":"ContainerStarted","Data":"ed1ffd915c9ec2934c0c3b1e597f12ac950743b6b646f98ee1cafe0f4277adb6"}
Oct 31 22:00:02.575352 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.574678    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j" event={"ID":"0a11ccff-84e3-4179-9f19-f07b86783bd3","Type":"ContainerStarted","Data":"557a591814a9d435de06d852ca477c9bb6fa5fdb87e09aa62576cb34d218932f"}
Oct 31 22:00:02.575352 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.574736    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j" event={"ID":"0a11ccff-84e3-4179-9f19-f07b86783bd3","Type":"ContainerStarted","Data":"6a64122fc869a80fa3000cac008c0f744419f6324ee706ca762d63b11a8398c5"}
Oct 31 22:00:02.601535 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.601474    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg" podStartSLOduration=2.601452465 podStartE2EDuration="2.601452465s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:00:02.595672599 +0000 UTC m=+2642.511982036" watchObservedRunningTime="2026-10-31 22:00:02.601452465 +0000 UTC m=+2642.517761896"
Oct 31 22:00:02.640312 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.640277    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj"]
Oct 31 22:00:02.640832 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.640777    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk" podStartSLOduration=2.640758391 podStartE2EDuration="2.640758391s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:00:02.638286363 +0000 UTC m=+2642.554595795" watchObservedRunningTime="2026-10-31 22:00:02.640758391 +0000 UTC m=+2642.557067822"
Oct 31 22:00:02.641580 ip-10-0-136-157 kubenswrapper[4460]: W1031 22:00:02.641554    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69ea00be_260e_490a_bafd_e629d23f57cb.slice/crio-c3f5ab30bcce2ccbb06a52d98fdda6cb4cbb8175f87cce7e93b3d322f979831b WatchSource:0}: Error finding container c3f5ab30bcce2ccbb06a52d98fdda6cb4cbb8175f87cce7e93b3d322f979831b: Status 404 returned error can't find the container with id c3f5ab30bcce2ccbb06a52d98fdda6cb4cbb8175f87cce7e93b3d322f979831b
Oct 31 22:00:02.735338 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.735268    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n" podStartSLOduration=2.735249149 podStartE2EDuration="2.735249149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:00:02.729256823 +0000 UTC m=+2642.645566253" watchObservedRunningTime="2026-10-31 22:00:02.735249149 +0000 UTC m=+2642.651558582"
Oct 31 22:00:02.777775 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.777717    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r" podStartSLOduration=2.777699832 podStartE2EDuration="2.777699832s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:00:02.773572634 +0000 UTC m=+2642.689882065" watchObservedRunningTime="2026-10-31 22:00:02.777699832 +0000 UTC m=+2642.694009307"
Oct 31 22:00:02.820463 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.820327    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w" podStartSLOduration=2.820308803 podStartE2EDuration="2.820308803s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:00:02.817101839 +0000 UTC m=+2642.733411269" watchObservedRunningTime="2026-10-31 22:00:02.820308803 +0000 UTC m=+2642.736618233"
Oct 31 22:00:02.908492 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.908435    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w" podStartSLOduration=2.908418378 podStartE2EDuration="2.908418378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:00:02.904563208 +0000 UTC m=+2642.820872638" watchObservedRunningTime="2026-10-31 22:00:02.908418378 +0000 UTC m=+2642.824727808"
Oct 31 22:00:02.952977 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.952900    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj" podStartSLOduration=2.952885981 podStartE2EDuration="2.952885981s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:00:02.947907795 +0000 UTC m=+2642.864217224" watchObservedRunningTime="2026-10-31 22:00:02.952885981 +0000 UTC m=+2642.869195411"
Oct 31 22:00:02.994613 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:02.994556    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j" podStartSLOduration=2.9945427430000002 podStartE2EDuration="2.994542743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:00:02.990656976 +0000 UTC m=+2642.906966406" watchObservedRunningTime="2026-10-31 22:00:02.994542743 +0000 UTC m=+2642.910852172"
Oct 31 22:00:03.037812 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.037758    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f" podStartSLOduration=3.037743901 podStartE2EDuration="3.037743901s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:00:03.035647889 +0000 UTC m=+2642.951957320" watchObservedRunningTime="2026-10-31 22:00:03.037743901 +0000 UTC m=+2642.954053331"
Oct 31 22:00:03.144440 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.144401    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"]
Oct 31 22:00:03.144701 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.144676    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj" podUID="d8dbe7b2-34ce-4590-825e-36a108f86b9f" containerName="istio-proxy" containerID="cri-o://0e09d03a72a23024608dc7d8a250b7df63acdf799cd131571b9cd00f082cefd0" gracePeriod=30
Oct 31 22:00:03.584425 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.584336    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj" event={"ID":"69ea00be-260e-490a-bafd-e629d23f57cb","Type":"ContainerStarted","Data":"0d2b60c2fb8824387797d0c1f91f7902def6024a62b3362d2b7cd33a3de92be6"}
Oct 31 22:00:03.584425 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.584387    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj" event={"ID":"69ea00be-260e-490a-bafd-e629d23f57cb","Type":"ContainerStarted","Data":"c3f5ab30bcce2ccbb06a52d98fdda6cb4cbb8175f87cce7e93b3d322f979831b"}
Oct 31 22:00:03.584872 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.584437    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj"
Oct 31 22:00:03.586517 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.586481    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq" event={"ID":"229deecb-4066-4ef7-bcf0-70bd5c15df45","Type":"ContainerStarted","Data":"5f79a89387fc71f3287426d4e32aa7d295b67f09e3de0eaed4b4646ecc248431"}
Oct 31 22:00:03.588933 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.588896    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk" event={"ID":"31d27730-27b7-4bfa-ac38-5a4a3cc13487","Type":"ContainerStarted","Data":"844d809b760b2d4cb74d2cd1a9981ca7819b05e23b14421ec64a73945866a693"}
Oct 31 22:00:03.590760 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.590734    4460 generic.go:356] "Generic (PLEG): container finished" podID="63aa3078-082d-400e-abfb-9985668831fc" containerID="31306d2ac046511759a5fa2eb07103a04cc93cf72f613521c565f512b6b68a53" exitCode=60
Oct 31 22:00:03.590888 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.590802    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w" event={"ID":"63aa3078-082d-400e-abfb-9985668831fc","Type":"ContainerDied","Data":"31306d2ac046511759a5fa2eb07103a04cc93cf72f613521c565f512b6b68a53"}
Oct 31 22:00:03.606928 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.591125    4460 scope.go:122] "RemoveContainer" containerID="31306d2ac046511759a5fa2eb07103a04cc93cf72f613521c565f512b6b68a53"
Oct 31 22:00:03.606928 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.592754    4460 generic.go:356] "Generic (PLEG): container finished" podID="fb824101-1f12-403b-bf06-1051f6f25d17" containerID="fa5878739604c3e1a008c0c4d572c93605209d81494c0b964ae937a3b7007767" exitCode=60
Oct 31 22:00:03.606928 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.592835    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n" event={"ID":"fb824101-1f12-403b-bf06-1051f6f25d17","Type":"ContainerDied","Data":"fa5878739604c3e1a008c0c4d572c93605209d81494c0b964ae937a3b7007767"}
Oct 31 22:00:03.606928 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.593052    4460 scope.go:122] "RemoveContainer" containerID="fa5878739604c3e1a008c0c4d572c93605209d81494c0b964ae937a3b7007767"
Oct 31 22:00:03.606928 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.594893    4460 generic.go:356] "Generic (PLEG): container finished" podID="e081a034-3ce7-4af9-ba84-d44f463ae8fe" containerID="e10a1d6e9264c4d9922a5612c82100b6ef0d3c0a5b7eed4985ea68424e4c52f3" exitCode=60
Oct 31 22:00:03.606928 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.594923    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj" event={"ID":"e081a034-3ce7-4af9-ba84-d44f463ae8fe","Type":"ContainerDied","Data":"e10a1d6e9264c4d9922a5612c82100b6ef0d3c0a5b7eed4985ea68424e4c52f3"}
Oct 31 22:00:03.606928 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.595221    4460 scope.go:122] "RemoveContainer" containerID="e10a1d6e9264c4d9922a5612c82100b6ef0d3c0a5b7eed4985ea68424e4c52f3"
Oct 31 22:00:03.606928 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.598911    4460 generic.go:356] "Generic (PLEG): container finished" podID="093b9d9a-ec69-4552-a889-6f3bccda27cc" containerID="539d96f110256cf72bdb80ab7dfac478f9d21f50f999d9f11f63866b5c108ab6" exitCode=60
Oct 31 22:00:03.606928 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.598968    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r" event={"ID":"093b9d9a-ec69-4552-a889-6f3bccda27cc","Type":"ContainerDied","Data":"539d96f110256cf72bdb80ab7dfac478f9d21f50f999d9f11f63866b5c108ab6"}
Oct 31 22:00:03.606928 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.599229    4460 scope.go:122] "RemoveContainer" containerID="539d96f110256cf72bdb80ab7dfac478f9d21f50f999d9f11f63866b5c108ab6"
Oct 31 22:00:03.606928 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.600868    4460 generic.go:356] "Generic (PLEG): container finished" podID="0864e283-2018-4c62-abeb-809f9dd12ce4" containerID="81a70702a7a2d04b6ae469085e9e5c4f2252661d14160b937b80337ad0ddbd32" exitCode=60
Oct 31 22:00:03.606928 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.600978    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w" event={"ID":"0864e283-2018-4c62-abeb-809f9dd12ce4","Type":"ContainerDied","Data":"81a70702a7a2d04b6ae469085e9e5c4f2252661d14160b937b80337ad0ddbd32"}
Oct 31 22:00:03.606928 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.601168    4460 scope.go:122] "RemoveContainer" containerID="81a70702a7a2d04b6ae469085e9e5c4f2252661d14160b937b80337ad0ddbd32"
Oct 31 22:00:03.606928 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.603302    4460 generic.go:356] "Generic (PLEG): container finished" podID="23b20e87-150e-48d7-9da9-d6c81de609d7" containerID="0a26bcfd162255b4f9059392378c582608184246534700408974c77e748178c3" exitCode=60
Oct 31 22:00:03.606928 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.603412    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk" event={"ID":"23b20e87-150e-48d7-9da9-d6c81de609d7","Type":"ContainerDied","Data":"0a26bcfd162255b4f9059392378c582608184246534700408974c77e748178c3"}
Oct 31 22:00:03.606928 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.603613    4460 scope.go:122] "RemoveContainer" containerID="0a26bcfd162255b4f9059392378c582608184246534700408974c77e748178c3"
Oct 31 22:00:03.606928 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.606670    4460 generic.go:356] "Generic (PLEG): container finished" podID="76255444-0212-46b9-9a6d-571bc0ba8f28" containerID="d960761c0c7638d9d0a06aa39332bcf031bdfe7929135921352ff9a7259cf09d" exitCode=60
Oct 31 22:00:03.606928 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.606743    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f" event={"ID":"76255444-0212-46b9-9a6d-571bc0ba8f28","Type":"ContainerDied","Data":"d960761c0c7638d9d0a06aa39332bcf031bdfe7929135921352ff9a7259cf09d"}
Oct 31 22:00:03.609116 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.606959    4460 scope.go:122] "RemoveContainer" containerID="d960761c0c7638d9d0a06aa39332bcf031bdfe7929135921352ff9a7259cf09d"
Oct 31 22:00:03.609116 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.608787    4460 generic.go:356] "Generic (PLEG): container finished" podID="0a11ccff-84e3-4179-9f19-f07b86783bd3" containerID="557a591814a9d435de06d852ca477c9bb6fa5fdb87e09aa62576cb34d218932f" exitCode=60
Oct 31 22:00:03.609116 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.608818    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j" event={"ID":"0a11ccff-84e3-4179-9f19-f07b86783bd3","Type":"ContainerDied","Data":"557a591814a9d435de06d852ca477c9bb6fa5fdb87e09aa62576cb34d218932f"}
Oct 31 22:00:03.609116 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.609079    4460 scope.go:122] "RemoveContainer" containerID="557a591814a9d435de06d852ca477c9bb6fa5fdb87e09aa62576cb34d218932f"
Oct 31 22:00:03.641681 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.641620    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj" podStartSLOduration=2.641600915 podStartE2EDuration="2.641600915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:00:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:00:03.639140437 +0000 UTC m=+2643.555449868" watchObservedRunningTime="2026-10-31 22:00:03.641600915 +0000 UTC m=+2643.557910346"
Oct 31 22:00:03.735891 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.735841    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk" podStartSLOduration=3.735827007 podStartE2EDuration="3.735827007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:00:03.690750005 +0000 UTC m=+2643.607059435" watchObservedRunningTime="2026-10-31 22:00:03.735827007 +0000 UTC m=+2643.652136437"
Oct 31 22:00:03.957882 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:03.957824    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq" podStartSLOduration=3.957810669 podStartE2EDuration="3.957810669s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:00:03.956091518 +0000 UTC m=+2643.872400953" watchObservedRunningTime="2026-10-31 22:00:03.957810669 +0000 UTC m=+2643.874120132"
Oct 31 22:00:04.090040 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:04.090006    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:04.618853 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:04.618811    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w" event={"ID":"0864e283-2018-4c62-abeb-809f9dd12ce4","Type":"ContainerStarted","Data":"8f3d52322fead80e3e8a2c94fe48c31f052cd8141ef433bc6bea8eeec9606482"}
Oct 31 22:00:04.624669 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:04.624632    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk" event={"ID":"23b20e87-150e-48d7-9da9-d6c81de609d7","Type":"ContainerStarted","Data":"e9a03749a1f08964d41deadc867f736e7e2ab1024e5cafc2d9c1152750aa5a13"}
Oct 31 22:00:04.627010 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:04.626976    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f" event={"ID":"76255444-0212-46b9-9a6d-571bc0ba8f28","Type":"ContainerStarted","Data":"9738abce6b4502079a1e58f0d8faa866483af01d3fafa94ff4c9169391e95a7d"}
Oct 31 22:00:04.629212 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:04.629177    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j" event={"ID":"0a11ccff-84e3-4179-9f19-f07b86783bd3","Type":"ContainerStarted","Data":"d786bffa958f14b2a588a1b092a3ebb7832980548101598bd1695f4ccb5a50f0"}
Oct 31 22:00:04.631337 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:04.631313    4460 generic.go:356] "Generic (PLEG): container finished" podID="229deecb-4066-4ef7-bcf0-70bd5c15df45" containerID="5f79a89387fc71f3287426d4e32aa7d295b67f09e3de0eaed4b4646ecc248431" exitCode=60
Oct 31 22:00:04.631456 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:04.631367    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq" event={"ID":"229deecb-4066-4ef7-bcf0-70bd5c15df45","Type":"ContainerDied","Data":"5f79a89387fc71f3287426d4e32aa7d295b67f09e3de0eaed4b4646ecc248431"}
Oct 31 22:00:04.631456 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:04.631418    4460 scope.go:122] "RemoveContainer" containerID="183fd1fd41664c6467682918b9d6c98495cdff27fb79509ecdb5a24e8591934c"
Oct 31 22:00:04.631611 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:04.631595    4460 scope.go:122] "RemoveContainer" containerID="5f79a89387fc71f3287426d4e32aa7d295b67f09e3de0eaed4b4646ecc248431"
Oct 31 22:00:04.631917 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:04.631894    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq_odh-ai-gateway-infra(229deecb-4066-4ef7-bcf0-70bd5c15df45)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq" podUID="229deecb-4066-4ef7-bcf0-70bd5c15df45"
Oct 31 22:00:04.633365 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:04.633340    4460 generic.go:356] "Generic (PLEG): container finished" podID="31d27730-27b7-4bfa-ac38-5a4a3cc13487" containerID="844d809b760b2d4cb74d2cd1a9981ca7819b05e23b14421ec64a73945866a693" exitCode=60
Oct 31 22:00:04.633487 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:04.633423    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk" event={"ID":"31d27730-27b7-4bfa-ac38-5a4a3cc13487","Type":"ContainerDied","Data":"844d809b760b2d4cb74d2cd1a9981ca7819b05e23b14421ec64a73945866a693"}
Oct 31 22:00:04.633628 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:04.633612    4460 scope.go:122] "RemoveContainer" containerID="844d809b760b2d4cb74d2cd1a9981ca7819b05e23b14421ec64a73945866a693"
Oct 31 22:00:04.633884 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:04.633865    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk_odh-ai-gateway-infra(31d27730-27b7-4bfa-ac38-5a4a3cc13487)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk" podUID="31d27730-27b7-4bfa-ac38-5a4a3cc13487"
Oct 31 22:00:04.635504 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:04.635477    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w" event={"ID":"63aa3078-082d-400e-abfb-9985668831fc","Type":"ContainerStarted","Data":"899d86b8923e4c31b1c37e65e619f7c2170aaa2a6ff934bb0b7f852ad35674e2"}
Oct 31 22:00:04.637511 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:04.637487    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n" event={"ID":"fb824101-1f12-403b-bf06-1051f6f25d17","Type":"ContainerStarted","Data":"d4765fe3e227cbeb4c080abd3d337575e2a20eadb941ce8947ae6f716f057d8f"}
Oct 31 22:00:04.639554 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:04.639527    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj" event={"ID":"e081a034-3ce7-4af9-ba84-d44f463ae8fe","Type":"ContainerStarted","Data":"3b3f24b9b111142a25904c4b929c109dc8093dff991be52665aaf52b5c478fcf"}
Oct 31 22:00:04.641627 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:04.641594    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r" event={"ID":"093b9d9a-ec69-4552-a889-6f3bccda27cc","Type":"ContainerStarted","Data":"c6a8f7e495c112c9344441d446700ae5ade9b9890e18f456edf6ea9b1453b2b4"}
Oct 31 22:00:04.646317 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:04.646296    4460 scope.go:122] "RemoveContainer" containerID="50ba29ebdf8ffc0b381a2efdd77dc226c4a27802344ccc9ef57aa865406c890e"
Oct 31 22:00:05.651860 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.651825    4460 generic.go:356] "Generic (PLEG): container finished" podID="093b9d9a-ec69-4552-a889-6f3bccda27cc" containerID="c6a8f7e495c112c9344441d446700ae5ade9b9890e18f456edf6ea9b1453b2b4" exitCode=60
Oct 31 22:00:05.652374 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.651897    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r" event={"ID":"093b9d9a-ec69-4552-a889-6f3bccda27cc","Type":"ContainerDied","Data":"c6a8f7e495c112c9344441d446700ae5ade9b9890e18f456edf6ea9b1453b2b4"}
Oct 31 22:00:05.652374 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.651968    4460 scope.go:122] "RemoveContainer" containerID="539d96f110256cf72bdb80ab7dfac478f9d21f50f999d9f11f63866b5c108ab6"
Oct 31 22:00:05.652374 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.652324    4460 scope.go:122] "RemoveContainer" containerID="c6a8f7e495c112c9344441d446700ae5ade9b9890e18f456edf6ea9b1453b2b4"
Oct 31 22:00:05.652769 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:05.652740    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r_odh-ai-gateway-infra(093b9d9a-ec69-4552-a889-6f3bccda27cc)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r" podUID="093b9d9a-ec69-4552-a889-6f3bccda27cc"
Oct 31 22:00:05.654086 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.654065    4460 generic.go:356] "Generic (PLEG): container finished" podID="0864e283-2018-4c62-abeb-809f9dd12ce4" containerID="8f3d52322fead80e3e8a2c94fe48c31f052cd8141ef433bc6bea8eeec9606482" exitCode=60
Oct 31 22:00:05.654179 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.654132    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w" event={"ID":"0864e283-2018-4c62-abeb-809f9dd12ce4","Type":"ContainerDied","Data":"8f3d52322fead80e3e8a2c94fe48c31f052cd8141ef433bc6bea8eeec9606482"}
Oct 31 22:00:05.654442 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.654419    4460 scope.go:122] "RemoveContainer" containerID="8f3d52322fead80e3e8a2c94fe48c31f052cd8141ef433bc6bea8eeec9606482"
Oct 31 22:00:05.654761 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:05.654737    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w_odh-ai-gateway-infra(0864e283-2018-4c62-abeb-809f9dd12ce4)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w" podUID="0864e283-2018-4c62-abeb-809f9dd12ce4"
Oct 31 22:00:05.656115 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.656093    4460 generic.go:356] "Generic (PLEG): container finished" podID="23b20e87-150e-48d7-9da9-d6c81de609d7" containerID="e9a03749a1f08964d41deadc867f736e7e2ab1024e5cafc2d9c1152750aa5a13" exitCode=60
Oct 31 22:00:05.656224 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.656123    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk" event={"ID":"23b20e87-150e-48d7-9da9-d6c81de609d7","Type":"ContainerDied","Data":"e9a03749a1f08964d41deadc867f736e7e2ab1024e5cafc2d9c1152750aa5a13"}
Oct 31 22:00:05.656438 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.656422    4460 scope.go:122] "RemoveContainer" containerID="e9a03749a1f08964d41deadc867f736e7e2ab1024e5cafc2d9c1152750aa5a13"
Oct 31 22:00:05.656737 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:05.656717    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk_odh-ai-gateway-infra(23b20e87-150e-48d7-9da9-d6c81de609d7)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk" podUID="23b20e87-150e-48d7-9da9-d6c81de609d7"
Oct 31 22:00:05.658204 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.658183    4460 generic.go:356] "Generic (PLEG): container finished" podID="76255444-0212-46b9-9a6d-571bc0ba8f28" containerID="9738abce6b4502079a1e58f0d8faa866483af01d3fafa94ff4c9169391e95a7d" exitCode=60
Oct 31 22:00:05.658301 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.658262    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f" event={"ID":"76255444-0212-46b9-9a6d-571bc0ba8f28","Type":"ContainerDied","Data":"9738abce6b4502079a1e58f0d8faa866483af01d3fafa94ff4c9169391e95a7d"}
Oct 31 22:00:05.658475 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.658460    4460 scope.go:122] "RemoveContainer" containerID="9738abce6b4502079a1e58f0d8faa866483af01d3fafa94ff4c9169391e95a7d"
Oct 31 22:00:05.658729 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:05.658712    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f_odh-ai-gateway-infra(76255444-0212-46b9-9a6d-571bc0ba8f28)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f" podUID="76255444-0212-46b9-9a6d-571bc0ba8f28"
Oct 31 22:00:05.660433 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.660412    4460 generic.go:356] "Generic (PLEG): container finished" podID="0a11ccff-84e3-4179-9f19-f07b86783bd3" containerID="d786bffa958f14b2a588a1b092a3ebb7832980548101598bd1695f4ccb5a50f0" exitCode=60
Oct 31 22:00:05.660544 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.660433    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j" event={"ID":"0a11ccff-84e3-4179-9f19-f07b86783bd3","Type":"ContainerDied","Data":"d786bffa958f14b2a588a1b092a3ebb7832980548101598bd1695f4ccb5a50f0"}
Oct 31 22:00:05.660727 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.660711    4460 scope.go:122] "RemoveContainer" containerID="d786bffa958f14b2a588a1b092a3ebb7832980548101598bd1695f4ccb5a50f0"
Oct 31 22:00:05.660998 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:05.660976    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j_odh-ai-gateway-infra(0a11ccff-84e3-4179-9f19-f07b86783bd3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j" podUID="0a11ccff-84e3-4179-9f19-f07b86783bd3"
Oct 31 22:00:05.662362 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.662344    4460 scope.go:122] "RemoveContainer" containerID="5f79a89387fc71f3287426d4e32aa7d295b67f09e3de0eaed4b4646ecc248431"
Oct 31 22:00:05.662570 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:05.662549    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq_odh-ai-gateway-infra(229deecb-4066-4ef7-bcf0-70bd5c15df45)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq" podUID="229deecb-4066-4ef7-bcf0-70bd5c15df45"
Oct 31 22:00:05.663887 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.663869    4460 scope.go:122] "RemoveContainer" containerID="844d809b760b2d4cb74d2cd1a9981ca7819b05e23b14421ec64a73945866a693"
Oct 31 22:00:05.664127 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:05.664103    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk_odh-ai-gateway-infra(31d27730-27b7-4bfa-ac38-5a4a3cc13487)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk" podUID="31d27730-27b7-4bfa-ac38-5a4a3cc13487"
Oct 31 22:00:05.665676 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.665656    4460 generic.go:356] "Generic (PLEG): container finished" podID="63aa3078-082d-400e-abfb-9985668831fc" containerID="899d86b8923e4c31b1c37e65e619f7c2170aaa2a6ff934bb0b7f852ad35674e2" exitCode=60
Oct 31 22:00:05.665760 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.665730    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w" event={"ID":"63aa3078-082d-400e-abfb-9985668831fc","Type":"ContainerDied","Data":"899d86b8923e4c31b1c37e65e619f7c2170aaa2a6ff934bb0b7f852ad35674e2"}
Oct 31 22:00:05.666001 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.665980    4460 scope.go:122] "RemoveContainer" containerID="899d86b8923e4c31b1c37e65e619f7c2170aaa2a6ff934bb0b7f852ad35674e2"
Oct 31 22:00:05.666289 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:05.666266    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w_odh-ai-gateway-infra(63aa3078-082d-400e-abfb-9985668831fc)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w" podUID="63aa3078-082d-400e-abfb-9985668831fc"
Oct 31 22:00:05.667788 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.667764    4460 generic.go:356] "Generic (PLEG): container finished" podID="fb824101-1f12-403b-bf06-1051f6f25d17" containerID="d4765fe3e227cbeb4c080abd3d337575e2a20eadb941ce8947ae6f716f057d8f" exitCode=60
Oct 31 22:00:05.667883 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.667840    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n" event={"ID":"fb824101-1f12-403b-bf06-1051f6f25d17","Type":"ContainerDied","Data":"d4765fe3e227cbeb4c080abd3d337575e2a20eadb941ce8947ae6f716f057d8f"}
Oct 31 22:00:05.668070 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.668026    4460 scope.go:122] "RemoveContainer" containerID="81a70702a7a2d04b6ae469085e9e5c4f2252661d14160b937b80337ad0ddbd32"
Oct 31 22:00:05.668127 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.668078    4460 scope.go:122] "RemoveContainer" containerID="d4765fe3e227cbeb4c080abd3d337575e2a20eadb941ce8947ae6f716f057d8f"
Oct 31 22:00:05.668345 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:05.668317    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n_odh-ai-gateway-infra(fb824101-1f12-403b-bf06-1051f6f25d17)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n" podUID="fb824101-1f12-403b-bf06-1051f6f25d17"
Oct 31 22:00:05.669924 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.669907    4460 generic.go:356] "Generic (PLEG): container finished" podID="e081a034-3ce7-4af9-ba84-d44f463ae8fe" containerID="3b3f24b9b111142a25904c4b929c109dc8093dff991be52665aaf52b5c478fcf" exitCode=60
Oct 31 22:00:05.670019 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.669959    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj" event={"ID":"e081a034-3ce7-4af9-ba84-d44f463ae8fe","Type":"ContainerDied","Data":"3b3f24b9b111142a25904c4b929c109dc8093dff991be52665aaf52b5c478fcf"}
Oct 31 22:00:05.670193 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.670179    4460 scope.go:122] "RemoveContainer" containerID="3b3f24b9b111142a25904c4b929c109dc8093dff991be52665aaf52b5c478fcf"
Oct 31 22:00:05.670395 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:05.670378    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj_odh-ai-gateway-infra(e081a034-3ce7-4af9-ba84-d44f463ae8fe)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj" podUID="e081a034-3ce7-4af9-ba84-d44f463ae8fe"
Oct 31 22:00:05.682461 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.682437    4460 scope.go:122] "RemoveContainer" containerID="0a26bcfd162255b4f9059392378c582608184246534700408974c77e748178c3"
Oct 31 22:00:05.697972 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.697927    4460 scope.go:122] "RemoveContainer" containerID="d960761c0c7638d9d0a06aa39332bcf031bdfe7929135921352ff9a7259cf09d"
Oct 31 22:00:05.715928 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.715899    4460 scope.go:122] "RemoveContainer" containerID="557a591814a9d435de06d852ca477c9bb6fa5fdb87e09aa62576cb34d218932f"
Oct 31 22:00:05.732386 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.732363    4460 scope.go:122] "RemoveContainer" containerID="31306d2ac046511759a5fa2eb07103a04cc93cf72f613521c565f512b6b68a53"
Oct 31 22:00:05.746861 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.746834    4460 scope.go:122] "RemoveContainer" containerID="fa5878739604c3e1a008c0c4d572c93605209d81494c0b964ae937a3b7007767"
Oct 31 22:00:05.763014 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.762630    4460 scope.go:122] "RemoveContainer" containerID="e10a1d6e9264c4d9922a5612c82100b6ef0d3c0a5b7eed4985ea68424e4c52f3"
Oct 31 22:00:05.843676 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:05.843643    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:06.679571 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:06.679531    4460 scope.go:122] "RemoveContainer" containerID="c6a8f7e495c112c9344441d446700ae5ade9b9890e18f456edf6ea9b1453b2b4"
Oct 31 22:00:06.680074 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:06.679884    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r_odh-ai-gateway-infra(093b9d9a-ec69-4552-a889-6f3bccda27cc)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r" podUID="093b9d9a-ec69-4552-a889-6f3bccda27cc"
Oct 31 22:00:06.681279 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:06.681257    4460 scope.go:122] "RemoveContainer" containerID="8f3d52322fead80e3e8a2c94fe48c31f052cd8141ef433bc6bea8eeec9606482"
Oct 31 22:00:06.681527 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:06.681508    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w_odh-ai-gateway-infra(0864e283-2018-4c62-abeb-809f9dd12ce4)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w" podUID="0864e283-2018-4c62-abeb-809f9dd12ce4"
Oct 31 22:00:06.683037 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:06.683013    4460 scope.go:122] "RemoveContainer" containerID="e9a03749a1f08964d41deadc867f736e7e2ab1024e5cafc2d9c1152750aa5a13"
Oct 31 22:00:06.683290 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:06.683270    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk_odh-ai-gateway-infra(23b20e87-150e-48d7-9da9-d6c81de609d7)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk" podUID="23b20e87-150e-48d7-9da9-d6c81de609d7"
Oct 31 22:00:06.684586 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:06.684568    4460 scope.go:122] "RemoveContainer" containerID="9738abce6b4502079a1e58f0d8faa866483af01d3fafa94ff4c9169391e95a7d"
Oct 31 22:00:06.684816 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:06.684795    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f_odh-ai-gateway-infra(76255444-0212-46b9-9a6d-571bc0ba8f28)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f" podUID="76255444-0212-46b9-9a6d-571bc0ba8f28"
Oct 31 22:00:06.686048 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:06.686030    4460 scope.go:122] "RemoveContainer" containerID="d786bffa958f14b2a588a1b092a3ebb7832980548101598bd1695f4ccb5a50f0"
Oct 31 22:00:06.686267 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:06.686244    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j_odh-ai-gateway-infra(0a11ccff-84e3-4179-9f19-f07b86783bd3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j" podUID="0a11ccff-84e3-4179-9f19-f07b86783bd3"
Oct 31 22:00:06.687635 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:06.687617    4460 scope.go:122] "RemoveContainer" containerID="899d86b8923e4c31b1c37e65e619f7c2170aaa2a6ff934bb0b7f852ad35674e2"
Oct 31 22:00:06.687844 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:06.687818    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w_odh-ai-gateway-infra(63aa3078-082d-400e-abfb-9985668831fc)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w" podUID="63aa3078-082d-400e-abfb-9985668831fc"
Oct 31 22:00:06.689178 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:06.689163    4460 scope.go:122] "RemoveContainer" containerID="d4765fe3e227cbeb4c080abd3d337575e2a20eadb941ce8947ae6f716f057d8f"
Oct 31 22:00:06.689397 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:06.689376    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n_odh-ai-gateway-infra(fb824101-1f12-403b-bf06-1051f6f25d17)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n" podUID="fb824101-1f12-403b-bf06-1051f6f25d17"
Oct 31 22:00:06.690595 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:06.690580    4460 scope.go:122] "RemoveContainer" containerID="3b3f24b9b111142a25904c4b929c109dc8093dff991be52665aaf52b5c478fcf"
Oct 31 22:00:06.690765 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:06.690749    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj_odh-ai-gateway-infra(e081a034-3ce7-4af9-ba84-d44f463ae8fe)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj" podUID="e081a034-3ce7-4af9-ba84-d44f463ae8fe"
Oct 31 22:00:07.551588 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:07.551544    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg" podUID="7c2a2f28-904a-4aef-96dd-3165e1f391db" containerName="payload-processing" containerID="cri-o://6984e616eb3156e04d937be254dd97ea84a8db0efc0dfc74bb0c4aaba359d454" gracePeriod=25
Oct 31 22:00:07.636709 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:07.636675    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:07.706778 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:07.706748    4460 generic.go:356] "Generic (PLEG): container finished" podID="7c2a2f28-904a-4aef-96dd-3165e1f391db" containerID="6984e616eb3156e04d937be254dd97ea84a8db0efc0dfc74bb0c4aaba359d454" exitCode=0
Oct 31 22:00:07.707111 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:07.706820    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg" event={"ID":"7c2a2f28-904a-4aef-96dd-3165e1f391db","Type":"ContainerDied","Data":"6984e616eb3156e04d937be254dd97ea84a8db0efc0dfc74bb0c4aaba359d454"}
Oct 31 22:00:07.863949 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:07.863921    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg"
Oct 31 22:00:08.001283 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.001253    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7c2a2f28-904a-4aef-96dd-3165e1f391db-provider-credentials\" (UniqueName: \"kubernetes.io/projected/7c2a2f28-904a-4aef-96dd-3165e1f391db-provider-credentials\") pod \"7c2a2f28-904a-4aef-96dd-3165e1f391db\" (UID: \"7c2a2f28-904a-4aef-96dd-3165e1f391db\") "
Oct 31 22:00:08.001458 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.001299    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7c2a2f28-904a-4aef-96dd-3165e1f391db-kube-api-access-k99px\" (UniqueName: \"kubernetes.io/projected/7c2a2f28-904a-4aef-96dd-3165e1f391db-kube-api-access-k99px\") pod \"7c2a2f28-904a-4aef-96dd-3165e1f391db\" (UID: \"7c2a2f28-904a-4aef-96dd-3165e1f391db\") "
Oct 31 22:00:08.001540 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.001519    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7c2a2f28-904a-4aef-96dd-3165e1f391db-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7c2a2f28-904a-4aef-96dd-3165e1f391db-routing-overlay-volume\") pod \"7c2a2f28-904a-4aef-96dd-3165e1f391db\" (UID: \"7c2a2f28-904a-4aef-96dd-3165e1f391db\") "
Oct 31 22:00:08.001590 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.001574    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7c2a2f28-904a-4aef-96dd-3165e1f391db-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c2a2f28-904a-4aef-96dd-3165e1f391db-plugins-config-volume\") pod \"7c2a2f28-904a-4aef-96dd-3165e1f391db\" (UID: \"7c2a2f28-904a-4aef-96dd-3165e1f391db\") "
Oct 31 22:00:08.001911 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.001883    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c2a2f28-904a-4aef-96dd-3165e1f391db-routing-overlay-volume" pod "7c2a2f28-904a-4aef-96dd-3165e1f391db" (UID: "7c2a2f28-904a-4aef-96dd-3165e1f391db"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:00:08.002060 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.002013    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c2a2f28-904a-4aef-96dd-3165e1f391db-plugins-config-volume" pod "7c2a2f28-904a-4aef-96dd-3165e1f391db" (UID: "7c2a2f28-904a-4aef-96dd-3165e1f391db"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:00:08.002355 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.002334    4460 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7c2a2f28-904a-4aef-96dd-3165e1f391db-routing-overlay-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:08.002393 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.002364    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c2a2f28-904a-4aef-96dd-3165e1f391db-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:08.003557 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.003536    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c2a2f28-904a-4aef-96dd-3165e1f391db-kube-api-access-k99px" pod "7c2a2f28-904a-4aef-96dd-3165e1f391db" (UID: "7c2a2f28-904a-4aef-96dd-3165e1f391db"). InnerVolumeSpecName "kube-api-access-k99px". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:00:08.003640 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.003567    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c2a2f28-904a-4aef-96dd-3165e1f391db-provider-credentials" pod "7c2a2f28-904a-4aef-96dd-3165e1f391db" (UID: "7c2a2f28-904a-4aef-96dd-3165e1f391db"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:00:08.103775 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.103693    4460 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/7c2a2f28-904a-4aef-96dd-3165e1f391db-provider-credentials\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:08.103775 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.103723    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k99px\" (UniqueName: \"kubernetes.io/projected/7c2a2f28-904a-4aef-96dd-3165e1f391db-kube-api-access-k99px\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:08.275675 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.275648    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs"
Oct 31 22:00:08.401585 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.401559    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 22:00:08.407807 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.407782    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istio-envoy\") pod \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") "
Oct 31 22:00:08.408006 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.407932    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d8dbe7b2-34ce-4590-825e-36a108f86b9f-kube-api-access-m8gw2\" (UniqueName: \"kubernetes.io/projected/d8dbe7b2-34ce-4590-825e-36a108f86b9f-kube-api-access-m8gw2\") pod \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") "
Oct 31 22:00:08.408069 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.408035    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istio-podinfo\") pod \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") "
Oct 31 22:00:08.408104 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.408075    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istio-data\") pod \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") "
Oct 31 22:00:08.408155 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.408119    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-credential-socket\") pod \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") "
Oct 31 22:00:08.408211 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.408193    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istio-token\" (UniqueName: \"kubernetes.io/projected/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istio-token\") pod \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") "
Oct 31 22:00:08.408302 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.408285    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-workload-socket\") pod \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") "
Oct 31 22:00:08.408372 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.408346    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-workload-certs\") pod \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") "
Oct 31 22:00:08.408452 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.408406    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istio-data" pod "d8dbe7b2-34ce-4590-825e-36a108f86b9f" (UID: "d8dbe7b2-34ce-4590-825e-36a108f86b9f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:00:08.408508 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.408455    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istiod-ca-cert\") pod \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\" (UID: \"d8dbe7b2-34ce-4590-825e-36a108f86b9f\") "
Oct 31 22:00:08.408508 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.408478    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-credential-socket" pod "d8dbe7b2-34ce-4590-825e-36a108f86b9f" (UID: "d8dbe7b2-34ce-4590-825e-36a108f86b9f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:00:08.408639 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.408620    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-workload-socket" pod "d8dbe7b2-34ce-4590-825e-36a108f86b9f" (UID: "d8dbe7b2-34ce-4590-825e-36a108f86b9f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:00:08.408739 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.408717    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-workload-certs" pod "d8dbe7b2-34ce-4590-825e-36a108f86b9f" (UID: "d8dbe7b2-34ce-4590-825e-36a108f86b9f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:00:08.409059 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.409030    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istiod-ca-cert" pod "d8dbe7b2-34ce-4590-825e-36a108f86b9f" (UID: "d8dbe7b2-34ce-4590-825e-36a108f86b9f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:00:08.409209 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.409183    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:08.409209 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.409210    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:08.409385 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.409226    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:08.409385 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.409240    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:08.409385 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.409254    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:08.410344 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.410320    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istio-envoy" pod "d8dbe7b2-34ce-4590-825e-36a108f86b9f" (UID: "d8dbe7b2-34ce-4590-825e-36a108f86b9f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:00:08.410418 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.410338    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8dbe7b2-34ce-4590-825e-36a108f86b9f-kube-api-access-m8gw2" pod "d8dbe7b2-34ce-4590-825e-36a108f86b9f" (UID: "d8dbe7b2-34ce-4590-825e-36a108f86b9f"). InnerVolumeSpecName "kube-api-access-m8gw2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:00:08.410418 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.410319    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istio-podinfo" pod "d8dbe7b2-34ce-4590-825e-36a108f86b9f" (UID: "d8dbe7b2-34ce-4590-825e-36a108f86b9f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 22:00:08.410889 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.410872    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istio-token" pod "d8dbe7b2-34ce-4590-825e-36a108f86b9f" (UID: "d8dbe7b2-34ce-4590-825e-36a108f86b9f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:00:08.510352 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.510319    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:08.510352 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.510351    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8gw2\" (UniqueName: \"kubernetes.io/projected/d8dbe7b2-34ce-4590-825e-36a108f86b9f-kube-api-access-m8gw2\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:08.510568 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.510361    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:08.510568 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.510372    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8dbe7b2-34ce-4590-825e-36a108f86b9f-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:08.715751 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.715663    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg"
Oct 31 22:00:08.715751 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.715700    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg" event={"ID":"7c2a2f28-904a-4aef-96dd-3165e1f391db","Type":"ContainerDied","Data":"259ccfe6232c6a249ea3baa2e8ad6ce735f8123901fd2e3ac970c35bbd817669"}
Oct 31 22:00:08.716275 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.715761    4460 scope.go:122] "RemoveContainer" containerID="6984e616eb3156e04d937be254dd97ea84a8db0efc0dfc74bb0c4aaba359d454"
Oct 31 22:00:08.718017 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.717876    4460 generic.go:356] "Generic (PLEG): container finished" podID="d8dbe7b2-34ce-4590-825e-36a108f86b9f" containerID="0e09d03a72a23024608dc7d8a250b7df63acdf799cd131571b9cd00f082cefd0" exitCode=0
Oct 31 22:00:08.718145 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.718015    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj" event={"ID":"d8dbe7b2-34ce-4590-825e-36a108f86b9f","Type":"ContainerDied","Data":"0e09d03a72a23024608dc7d8a250b7df63acdf799cd131571b9cd00f082cefd0"}
Oct 31 22:00:08.718145 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.718065    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj" event={"ID":"d8dbe7b2-34ce-4590-825e-36a108f86b9f","Type":"ContainerDied","Data":"42a3b5ad8355c09173d59f9670bf70521b2b6decc7014df23f3370fe918388f9"}
Oct 31 22:00:08.718145 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.718019    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"
Oct 31 22:00:08.728336 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.728315    4460 scope.go:122] "RemoveContainer" containerID="0e09d03a72a23024608dc7d8a250b7df63acdf799cd131571b9cd00f082cefd0"
Oct 31 22:00:08.738878 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.738860    4460 scope.go:122] "RemoveContainer" containerID="0e09d03a72a23024608dc7d8a250b7df63acdf799cd131571b9cd00f082cefd0"
Oct 31 22:00:08.739169 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:08.739148    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e09d03a72a23024608dc7d8a250b7df63acdf799cd131571b9cd00f082cefd0\": container with ID starting with 0e09d03a72a23024608dc7d8a250b7df63acdf799cd131571b9cd00f082cefd0 not found: ID does not exist" containerID="0e09d03a72a23024608dc7d8a250b7df63acdf799cd131571b9cd00f082cefd0"
Oct 31 22:00:08.739246 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.739176    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e09d03a72a23024608dc7d8a250b7df63acdf799cd131571b9cd00f082cefd0"} err="failed to get container status \"0e09d03a72a23024608dc7d8a250b7df63acdf799cd131571b9cd00f082cefd0\": rpc error: code = NotFound desc = could not find container \"0e09d03a72a23024608dc7d8a250b7df63acdf799cd131571b9cd00f082cefd0\": container with ID starting with 0e09d03a72a23024608dc7d8a250b7df63acdf799cd131571b9cd00f082cefd0 not found: ID does not exist"
Oct 31 22:00:08.775322 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.775297    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg"]
Oct 31 22:00:08.780763 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.780743    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-66b42c7tg"]
Oct 31 22:00:08.829211 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.829180    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"]
Oct 31 22:00:08.836077 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:08.836044    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-v8swj"]
Oct 31 22:00:09.328898 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:09.328856    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:10.231405 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:10.231368    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7c2a2f28-904a-4aef-96dd-3165e1f391db" path="/var/lib/kubelet/pods/7c2a2f28-904a-4aef-96dd-3165e1f391db/volumes"
Oct 31 22:00:10.232156 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:10.232133    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d8dbe7b2-34ce-4590-825e-36a108f86b9f" path="/var/lib/kubelet/pods/d8dbe7b2-34ce-4590-825e-36a108f86b9f/volumes"
Oct 31 22:00:10.324390 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:10.324353    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:10.435650 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:10.435594    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:10.729769 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:10.729728    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8"]
Oct 31 22:00:10.730766 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:10.730746    4460 cpu_manager.go:414] "containerMap: removing container" podUID="7c2a2f28-904a-4aef-96dd-3165e1f391db" containerName="payload-processing"
Oct 31 22:00:10.730766 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:10.730766    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c2a2f28-904a-4aef-96dd-3165e1f391db" containerName="payload-processing"
Oct 31 22:00:10.730877 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:10.730792    4460 cpu_manager.go:414] "containerMap: removing container" podUID="d8dbe7b2-34ce-4590-825e-36a108f86b9f" containerName="istio-proxy"
Oct 31 22:00:10.730877 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:10.730797    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d8dbe7b2-34ce-4590-825e-36a108f86b9f" containerName="istio-proxy"
Oct 31 22:00:10.731001 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:10.730964    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="7c2a2f28-904a-4aef-96dd-3165e1f391db" containerName="payload-processing"
Oct 31 22:00:10.731001 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:10.730978    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="d8dbe7b2-34ce-4590-825e-36a108f86b9f" containerName="istio-proxy"
Oct 31 22:00:10.736752 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:10.736729    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8"
Oct 31 22:00:10.755068 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:10.755030    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8"]
Oct 31 22:00:10.762825 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:10.762790    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-4a73b1\"/\"payload-processing-external-model-plugins-e2e-extmod-b-4a73b1\""
Oct 31 22:00:10.763072 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:10.762838    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-4a73b1\"/\"kube-root-ca.crt\""
Oct 31 22:00:10.763250 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:10.762980    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-4a73b1\"/\"openshift-service-ca.crt\""
Oct 31 22:00:10.763363 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:10.763021    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-4a73b1\"/\"agc-cred-eb8c6251292d\""
Oct 31 22:00:10.763447 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:10.763074    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-4a73b1\"/\"agc-cred-da7a4611d93a\""
Oct 31 22:00:10.764440 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:10.764414    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-4a73b1\"/\"routing-overlay\""
Oct 31 22:00:10.766583 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:10.766560    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-4a73b1\"/\"payload-processing-external-model-e2e--23c252ba-dockercfg-l8pcd\""
Oct 31 22:00:10.876627 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:10.876589    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxnpd\" (UniqueName: \"kubernetes.io/projected/12241bee-ccf4-4a34-8e1a-cfec063d543e-kube-api-access-dxnpd\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8\" (UID: \"12241bee-ccf4-4a34-8e1a-cfec063d543e\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8"
Oct 31 22:00:10.876819 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:10.876777    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/12241bee-ccf4-4a34-8e1a-cfec063d543e-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8\" (UID: \"12241bee-ccf4-4a34-8e1a-cfec063d543e\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8"
Oct 31 22:00:10.876882 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:10.876840    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/12241bee-ccf4-4a34-8e1a-cfec063d543e-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8\" (UID: \"12241bee-ccf4-4a34-8e1a-cfec063d543e\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8"
Oct 31 22:00:10.876932 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:10.876915    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12241bee-ccf4-4a34-8e1a-cfec063d543e-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8\" (UID: \"12241bee-ccf4-4a34-8e1a-cfec063d543e\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8"
Oct 31 22:00:10.983211 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:10.983118    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/12241bee-ccf4-4a34-8e1a-cfec063d543e-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8\" (UID: \"12241bee-ccf4-4a34-8e1a-cfec063d543e\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8"
Oct 31 22:00:10.995962 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:10.983977    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/12241bee-ccf4-4a34-8e1a-cfec063d543e-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8\" (UID: \"12241bee-ccf4-4a34-8e1a-cfec063d543e\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8"
Oct 31 22:00:10.996250 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:10.996221    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12241bee-ccf4-4a34-8e1a-cfec063d543e-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8\" (UID: \"12241bee-ccf4-4a34-8e1a-cfec063d543e\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8"
Oct 31 22:00:10.996513 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:10.996494    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxnpd\" (UniqueName: \"kubernetes.io/projected/12241bee-ccf4-4a34-8e1a-cfec063d543e-kube-api-access-dxnpd\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8\" (UID: \"12241bee-ccf4-4a34-8e1a-cfec063d543e\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8"
Oct 31 22:00:10.997063 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:10.984779    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/12241bee-ccf4-4a34-8e1a-cfec063d543e-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8\" (UID: \"12241bee-ccf4-4a34-8e1a-cfec063d543e\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8"
Oct 31 22:00:11.019968 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:10.997984    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12241bee-ccf4-4a34-8e1a-cfec063d543e-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8\" (UID: \"12241bee-ccf4-4a34-8e1a-cfec063d543e\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8"
Oct 31 22:00:11.092513 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:11.092472    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxnpd\" (UniqueName: \"kubernetes.io/projected/12241bee-ccf4-4a34-8e1a-cfec063d543e-kube-api-access-dxnpd\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8\" (UID: \"12241bee-ccf4-4a34-8e1a-cfec063d543e\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8"
Oct 31 22:00:11.093255 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:11.093226    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/12241bee-ccf4-4a34-8e1a-cfec063d543e-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8\" (UID: \"12241bee-ccf4-4a34-8e1a-cfec063d543e\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8"
Oct 31 22:00:11.260711 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:11.260677    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:11.357820 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:11.357787    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8"
Oct 31 22:00:11.670315 ip-10-0-136-157 kubenswrapper[4460]: W1031 22:00:11.670281    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12241bee_ccf4_4a34_8e1a_cfec063d543e.slice/crio-faec8dbb4130776d4b75b70a23eb79c5839a1d6d3df55971d9b36dec673d64f2 WatchSource:0}: Error finding container faec8dbb4130776d4b75b70a23eb79c5839a1d6d3df55971d9b36dec673d64f2: Status 404 returned error can't find the container with id faec8dbb4130776d4b75b70a23eb79c5839a1d6d3df55971d9b36dec673d64f2
Oct 31 22:00:11.671502 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:11.671471    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8"]
Oct 31 22:00:11.753998 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:11.753967    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8" event={"ID":"12241bee-ccf4-4a34-8e1a-cfec063d543e","Type":"ContainerStarted","Data":"faec8dbb4130776d4b75b70a23eb79c5839a1d6d3df55971d9b36dec673d64f2"}
Oct 31 22:00:12.763647 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:12.763610    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8" event={"ID":"12241bee-ccf4-4a34-8e1a-cfec063d543e","Type":"ContainerStarted","Data":"6a587722563da71d6dbd424faf05b868f905be0e70b8dbd2530ad8443c770bef"}
Oct 31 22:00:12.764162 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:12.763706    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8"
Oct 31 22:00:12.810907 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:12.810860    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8" podStartSLOduration=2.810846132 podStartE2EDuration="2.810846132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:00:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:00:12.807662955 +0000 UTC m=+2652.723972386" watchObservedRunningTime="2026-10-31 22:00:12.810846132 +0000 UTC m=+2652.727155558"
Oct 31 22:00:12.947994 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:12.947868    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:14.630559 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:14.630529    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:14.645247 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:14.645211    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj"
Oct 31 22:00:14.726243 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:14.726206    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k"]
Oct 31 22:00:15.218354 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:15.218313    4460 scope.go:122] "RemoveContainer" containerID="9738abce6b4502079a1e58f0d8faa866483af01d3fafa94ff4c9169391e95a7d"
Oct 31 22:00:15.790609 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:15.790523    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f" event={"ID":"76255444-0212-46b9-9a6d-571bc0ba8f28","Type":"ContainerStarted","Data":"7527b7cce96261e027741fabb39d1f8945a89099cdc7e4ae4f68fdba0acf94d7"}
Oct 31 22:00:16.217592 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:16.217560    4460 scope.go:122] "RemoveContainer" containerID="e9a03749a1f08964d41deadc867f736e7e2ab1024e5cafc2d9c1152750aa5a13"
Oct 31 22:00:16.217762 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:16.217704    4460 scope.go:122] "RemoveContainer" containerID="d4765fe3e227cbeb4c080abd3d337575e2a20eadb941ce8947ae6f716f057d8f"
Oct 31 22:00:16.217849 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:16.217819    4460 scope.go:122] "RemoveContainer" containerID="8f3d52322fead80e3e8a2c94fe48c31f052cd8141ef433bc6bea8eeec9606482"
Oct 31 22:00:16.218028 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:16.218005    4460 scope.go:122] "RemoveContainer" containerID="844d809b760b2d4cb74d2cd1a9981ca7819b05e23b14421ec64a73945866a693"
Oct 31 22:00:16.393992 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:16.393891    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:16.801725 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:16.801687    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk" event={"ID":"31d27730-27b7-4bfa-ac38-5a4a3cc13487","Type":"ContainerStarted","Data":"55f1f02b10f75f191c86eee5d5cc7601c65f0fbb2a119ffaf625a218e4169718"}
Oct 31 22:00:16.803858 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:16.803825    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n" event={"ID":"fb824101-1f12-403b-bf06-1051f6f25d17","Type":"ContainerStarted","Data":"520639a4bc156b9eb67d3e5fa66e2dc72900145d75be888a0abc1a3516820726"}
Oct 31 22:00:16.805919 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:16.805892    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w" event={"ID":"0864e283-2018-4c62-abeb-809f9dd12ce4","Type":"ContainerStarted","Data":"cdb4a08dfffa00604927c516fa792a0a9b6a4929270c79b8ff6ac951eb45747f"}
Oct 31 22:00:16.807802 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:16.807774    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk" event={"ID":"23b20e87-150e-48d7-9da9-d6c81de609d7","Type":"ContainerStarted","Data":"3657120c663bb820513b3d4382cd6845d72f962183a9319cf55aa212349bc012"}
Oct 31 22:00:16.809720 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:16.809698    4460 generic.go:356] "Generic (PLEG): container finished" podID="76255444-0212-46b9-9a6d-571bc0ba8f28" containerID="7527b7cce96261e027741fabb39d1f8945a89099cdc7e4ae4f68fdba0acf94d7" exitCode=60
Oct 31 22:00:16.809837 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:16.809769    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f" event={"ID":"76255444-0212-46b9-9a6d-571bc0ba8f28","Type":"ContainerDied","Data":"7527b7cce96261e027741fabb39d1f8945a89099cdc7e4ae4f68fdba0acf94d7"}
Oct 31 22:00:16.809837 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:16.809818    4460 scope.go:122] "RemoveContainer" containerID="9738abce6b4502079a1e58f0d8faa866483af01d3fafa94ff4c9169391e95a7d"
Oct 31 22:00:16.810076 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:16.810061    4460 scope.go:122] "RemoveContainer" containerID="7527b7cce96261e027741fabb39d1f8945a89099cdc7e4ae4f68fdba0acf94d7"
Oct 31 22:00:16.810309 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:16.810289    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f_odh-ai-gateway-infra(76255444-0212-46b9-9a6d-571bc0ba8f28)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f" podUID="76255444-0212-46b9-9a6d-571bc0ba8f28"
Oct 31 22:00:16.855029 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:16.854974    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f"]
Oct 31 22:00:17.218815 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:17.218780    4460 scope.go:122] "RemoveContainer" containerID="5f79a89387fc71f3287426d4e32aa7d295b67f09e3de0eaed4b4646ecc248431"
Oct 31 22:00:17.820034 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:17.819995    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq" event={"ID":"229deecb-4066-4ef7-bcf0-70bd5c15df45","Type":"ContainerStarted","Data":"55d2c879387bde7319cddfd037e9f503975dad200ecbf1699b9e3ed945fa626e"}
Oct 31 22:00:17.822240 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:17.822216    4460 generic.go:356] "Generic (PLEG): container finished" podID="31d27730-27b7-4bfa-ac38-5a4a3cc13487" containerID="55f1f02b10f75f191c86eee5d5cc7601c65f0fbb2a119ffaf625a218e4169718" exitCode=60
Oct 31 22:00:17.822352 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:17.822289    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk" event={"ID":"31d27730-27b7-4bfa-ac38-5a4a3cc13487","Type":"ContainerDied","Data":"55f1f02b10f75f191c86eee5d5cc7601c65f0fbb2a119ffaf625a218e4169718"}
Oct 31 22:00:17.822352 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:17.822346    4460 scope.go:122] "RemoveContainer" containerID="844d809b760b2d4cb74d2cd1a9981ca7819b05e23b14421ec64a73945866a693"
Oct 31 22:00:17.822738 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:17.822716    4460 scope.go:122] "RemoveContainer" containerID="55f1f02b10f75f191c86eee5d5cc7601c65f0fbb2a119ffaf625a218e4169718"
Oct 31 22:00:17.823134 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:17.823088    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk_odh-ai-gateway-infra(31d27730-27b7-4bfa-ac38-5a4a3cc13487)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk" podUID="31d27730-27b7-4bfa-ac38-5a4a3cc13487"
Oct 31 22:00:17.824452 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:17.824429    4460 generic.go:356] "Generic (PLEG): container finished" podID="fb824101-1f12-403b-bf06-1051f6f25d17" containerID="520639a4bc156b9eb67d3e5fa66e2dc72900145d75be888a0abc1a3516820726" exitCode=60
Oct 31 22:00:17.824565 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:17.824515    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n" event={"ID":"fb824101-1f12-403b-bf06-1051f6f25d17","Type":"ContainerDied","Data":"520639a4bc156b9eb67d3e5fa66e2dc72900145d75be888a0abc1a3516820726"}
Oct 31 22:00:17.824900 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:17.824878    4460 scope.go:122] "RemoveContainer" containerID="520639a4bc156b9eb67d3e5fa66e2dc72900145d75be888a0abc1a3516820726"
Oct 31 22:00:17.825241 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:17.825218    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n_odh-ai-gateway-infra(fb824101-1f12-403b-bf06-1051f6f25d17)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n" podUID="fb824101-1f12-403b-bf06-1051f6f25d17"
Oct 31 22:00:17.826600 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:17.826577    4460 generic.go:356] "Generic (PLEG): container finished" podID="0864e283-2018-4c62-abeb-809f9dd12ce4" containerID="cdb4a08dfffa00604927c516fa792a0a9b6a4929270c79b8ff6ac951eb45747f" exitCode=60
Oct 31 22:00:17.826688 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:17.826623    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w" event={"ID":"0864e283-2018-4c62-abeb-809f9dd12ce4","Type":"ContainerDied","Data":"cdb4a08dfffa00604927c516fa792a0a9b6a4929270c79b8ff6ac951eb45747f"}
Oct 31 22:00:17.826994 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:17.826976    4460 scope.go:122] "RemoveContainer" containerID="cdb4a08dfffa00604927c516fa792a0a9b6a4929270c79b8ff6ac951eb45747f"
Oct 31 22:00:17.827289 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:17.827270    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w_odh-ai-gateway-infra(0864e283-2018-4c62-abeb-809f9dd12ce4)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w" podUID="0864e283-2018-4c62-abeb-809f9dd12ce4"
Oct 31 22:00:17.828807 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:17.828788    4460 generic.go:356] "Generic (PLEG): container finished" podID="23b20e87-150e-48d7-9da9-d6c81de609d7" containerID="3657120c663bb820513b3d4382cd6845d72f962183a9319cf55aa212349bc012" exitCode=60
Oct 31 22:00:17.828892 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:17.828856    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk" event={"ID":"23b20e87-150e-48d7-9da9-d6c81de609d7","Type":"ContainerDied","Data":"3657120c663bb820513b3d4382cd6845d72f962183a9319cf55aa212349bc012"}
Oct 31 22:00:17.829237 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:17.829218    4460 scope.go:122] "RemoveContainer" containerID="3657120c663bb820513b3d4382cd6845d72f962183a9319cf55aa212349bc012"
Oct 31 22:00:17.829541 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:17.829515    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk_odh-ai-gateway-infra(23b20e87-150e-48d7-9da9-d6c81de609d7)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk" podUID="23b20e87-150e-48d7-9da9-d6c81de609d7"
Oct 31 22:00:17.837071 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:17.837046    4460 scope.go:122] "RemoveContainer" containerID="d4765fe3e227cbeb4c080abd3d337575e2a20eadb941ce8947ae6f716f057d8f"
Oct 31 22:00:17.852097 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:17.852071    4460 scope.go:122] "RemoveContainer" containerID="8f3d52322fead80e3e8a2c94fe48c31f052cd8141ef433bc6bea8eeec9606482"
Oct 31 22:00:17.858357 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:17.858328    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk"]
Oct 31 22:00:17.869327 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:17.869299    4460 scope.go:122] "RemoveContainer" containerID="e9a03749a1f08964d41deadc867f736e7e2ab1024e5cafc2d9c1152750aa5a13"
Oct 31 22:00:17.912139 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:17.911301    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n"]
Oct 31 22:00:17.947883 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:17.947853    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w"]
Oct 31 22:00:18.016085 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:18.016060    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f"
Oct 31 22:00:18.036091 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:18.036058    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/76255444-0212-46b9-9a6d-571bc0ba8f28-kube-api-access-wt5r7\" (UniqueName: \"kubernetes.io/projected/76255444-0212-46b9-9a6d-571bc0ba8f28-kube-api-access-wt5r7\") pod \"76255444-0212-46b9-9a6d-571bc0ba8f28\" (UID: \"76255444-0212-46b9-9a6d-571bc0ba8f28\") "
Oct 31 22:00:18.036403 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:18.036370    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/76255444-0212-46b9-9a6d-571bc0ba8f28-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/76255444-0212-46b9-9a6d-571bc0ba8f28-maas-api-service-ca\") pod \"76255444-0212-46b9-9a6d-571bc0ba8f28\" (UID: \"76255444-0212-46b9-9a6d-571bc0ba8f28\") "
Oct 31 22:00:18.036766 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:18.036726    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76255444-0212-46b9-9a6d-571bc0ba8f28-maas-api-service-ca" pod "76255444-0212-46b9-9a6d-571bc0ba8f28" (UID: "76255444-0212-46b9-9a6d-571bc0ba8f28"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:00:18.037032 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:18.037014    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/76255444-0212-46b9-9a6d-571bc0ba8f28-maas-api-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:18.038585 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:18.038558    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76255444-0212-46b9-9a6d-571bc0ba8f28-kube-api-access-wt5r7" pod "76255444-0212-46b9-9a6d-571bc0ba8f28" (UID: "76255444-0212-46b9-9a6d-571bc0ba8f28"). InnerVolumeSpecName "kube-api-access-wt5r7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:00:18.041011 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:18.040990    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk"]
Oct 31 22:00:18.131441 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:18.131408    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:18.138816 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:18.138789    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wt5r7\" (UniqueName: \"kubernetes.io/projected/76255444-0212-46b9-9a6d-571bc0ba8f28-kube-api-access-wt5r7\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:18.217757 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:18.217727    4460 scope.go:122] "RemoveContainer" containerID="c6a8f7e495c112c9344441d446700ae5ade9b9890e18f456edf6ea9b1453b2b4"
Oct 31 22:00:18.218102 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:18.218073    4460 scope.go:122] "RemoveContainer" containerID="d786bffa958f14b2a588a1b092a3ebb7832980548101598bd1695f4ccb5a50f0"
Oct 31 22:00:18.218664 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:18.218638    4460 scope.go:122] "RemoveContainer" containerID="3b3f24b9b111142a25904c4b929c109dc8093dff991be52665aaf52b5c478fcf"
Oct 31 22:00:18.845269 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:18.845238    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f"
Oct 31 22:00:18.845653 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:18.845278    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f" event={"ID":"76255444-0212-46b9-9a6d-571bc0ba8f28","Type":"ContainerDied","Data":"8e737645fe4f795f64652f49fdee7ac17608e98829a15c93cee1abc534662d07"}
Oct 31 22:00:18.845653 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:18.845341    4460 scope.go:122] "RemoveContainer" containerID="7527b7cce96261e027741fabb39d1f8945a89099cdc7e4ae4f68fdba0acf94d7"
Oct 31 22:00:18.848277 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:18.848250    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j" event={"ID":"0a11ccff-84e3-4179-9f19-f07b86783bd3","Type":"ContainerStarted","Data":"5fe99f96727506ccafbd1542c1a0b0473186408e9af80147ee066466583b3151"}
Oct 31 22:00:18.851234 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:18.851208    4460 generic.go:356] "Generic (PLEG): container finished" podID="229deecb-4066-4ef7-bcf0-70bd5c15df45" containerID="55d2c879387bde7319cddfd037e9f503975dad200ecbf1699b9e3ed945fa626e" exitCode=60
Oct 31 22:00:18.851338 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:18.851231    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq" event={"ID":"229deecb-4066-4ef7-bcf0-70bd5c15df45","Type":"ContainerDied","Data":"55d2c879387bde7319cddfd037e9f503975dad200ecbf1699b9e3ed945fa626e"}
Oct 31 22:00:18.851580 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:18.851558    4460 scope.go:122] "RemoveContainer" containerID="55d2c879387bde7319cddfd037e9f503975dad200ecbf1699b9e3ed945fa626e"
Oct 31 22:00:18.851822 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:18.851799    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq_odh-ai-gateway-infra(229deecb-4066-4ef7-bcf0-70bd5c15df45)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq" podUID="229deecb-4066-4ef7-bcf0-70bd5c15df45"
Oct 31 22:00:18.859068 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:18.859048    4460 scope.go:122] "RemoveContainer" containerID="5f79a89387fc71f3287426d4e32aa7d295b67f09e3de0eaed4b4646ecc248431"
Oct 31 22:00:18.859916 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:18.859197    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj" event={"ID":"e081a034-3ce7-4af9-ba84-d44f463ae8fe","Type":"ContainerStarted","Data":"977628f4d7e340c525bdb2e0abfff1392d35431c2f9b53dd7779271fbdbec952"}
Oct 31 22:00:18.861885 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:18.861859    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r" event={"ID":"093b9d9a-ec69-4552-a889-6f3bccda27cc","Type":"ContainerStarted","Data":"e1cd8cfd097f5953265e583ea38a965d1897a172c71de4fd72b795e0c53d0491"}
Oct 31 22:00:18.876803 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:18.876768    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq"]
Oct 31 22:00:19.081075 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.081042    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk"
Oct 31 22:00:19.153224 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.153183    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/23b20e87-150e-48d7-9da9-d6c81de609d7-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23b20e87-150e-48d7-9da9-d6c81de609d7-maas-api-service-ca\") pod \"23b20e87-150e-48d7-9da9-d6c81de609d7\" (UID: \"23b20e87-150e-48d7-9da9-d6c81de609d7\") "
Oct 31 22:00:19.153384 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.153314    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/23b20e87-150e-48d7-9da9-d6c81de609d7-kube-api-access-56cqw\" (UniqueName: \"kubernetes.io/projected/23b20e87-150e-48d7-9da9-d6c81de609d7-kube-api-access-56cqw\") pod \"23b20e87-150e-48d7-9da9-d6c81de609d7\" (UID: \"23b20e87-150e-48d7-9da9-d6c81de609d7\") "
Oct 31 22:00:19.153738 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.153704    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23b20e87-150e-48d7-9da9-d6c81de609d7-maas-api-service-ca" pod "23b20e87-150e-48d7-9da9-d6c81de609d7" (UID: "23b20e87-150e-48d7-9da9-d6c81de609d7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:00:19.154346 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.154136    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23b20e87-150e-48d7-9da9-d6c81de609d7-maas-api-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:19.159407 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.159374    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23b20e87-150e-48d7-9da9-d6c81de609d7-kube-api-access-56cqw" pod "23b20e87-150e-48d7-9da9-d6c81de609d7" (UID: "23b20e87-150e-48d7-9da9-d6c81de609d7"). InnerVolumeSpecName "kube-api-access-56cqw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:00:19.171386 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.171359    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk"
Oct 31 22:00:19.176422 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.176399    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w"
Oct 31 22:00:19.182989 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.182969    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n"
Oct 31 22:00:19.218033 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.217997    4460 scope.go:122] "RemoveContainer" containerID="899d86b8923e4c31b1c37e65e619f7c2170aaa2a6ff934bb0b7f852ad35674e2"
Oct 31 22:00:19.255347 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.255320    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0864e283-2018-4c62-abeb-809f9dd12ce4-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0864e283-2018-4c62-abeb-809f9dd12ce4-maas-api-service-ca\") pod \"0864e283-2018-4c62-abeb-809f9dd12ce4\" (UID: \"0864e283-2018-4c62-abeb-809f9dd12ce4\") "
Oct 31 22:00:19.255484 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.255414    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fb824101-1f12-403b-bf06-1051f6f25d17-kube-api-access-bbnc8\" (UniqueName: \"kubernetes.io/projected/fb824101-1f12-403b-bf06-1051f6f25d17-kube-api-access-bbnc8\") pod \"fb824101-1f12-403b-bf06-1051f6f25d17\" (UID: \"fb824101-1f12-403b-bf06-1051f6f25d17\") "
Oct 31 22:00:19.255484 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.255473    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0864e283-2018-4c62-abeb-809f9dd12ce4-kube-api-access-smjk8\" (UniqueName: \"kubernetes.io/projected/0864e283-2018-4c62-abeb-809f9dd12ce4-kube-api-access-smjk8\") pod \"0864e283-2018-4c62-abeb-809f9dd12ce4\" (UID: \"0864e283-2018-4c62-abeb-809f9dd12ce4\") "
Oct 31 22:00:19.255617 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.255551    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fb824101-1f12-403b-bf06-1051f6f25d17-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb824101-1f12-403b-bf06-1051f6f25d17-maas-api-service-ca\") pod \"fb824101-1f12-403b-bf06-1051f6f25d17\" (UID: \"fb824101-1f12-403b-bf06-1051f6f25d17\") "
Oct 31 22:00:19.255617 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.255599    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/31d27730-27b7-4bfa-ac38-5a4a3cc13487-kube-api-access-gt9rp\" (UniqueName: \"kubernetes.io/projected/31d27730-27b7-4bfa-ac38-5a4a3cc13487-kube-api-access-gt9rp\") pod \"31d27730-27b7-4bfa-ac38-5a4a3cc13487\" (UID: \"31d27730-27b7-4bfa-ac38-5a4a3cc13487\") "
Oct 31 22:00:19.255777 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.255707    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0864e283-2018-4c62-abeb-809f9dd12ce4-maas-api-service-ca" pod "0864e283-2018-4c62-abeb-809f9dd12ce4" (UID: "0864e283-2018-4c62-abeb-809f9dd12ce4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:00:19.255874 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.255720    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/31d27730-27b7-4bfa-ac38-5a4a3cc13487-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/31d27730-27b7-4bfa-ac38-5a4a3cc13487-maas-api-service-ca\") pod \"31d27730-27b7-4bfa-ac38-5a4a3cc13487\" (UID: \"31d27730-27b7-4bfa-ac38-5a4a3cc13487\") "
Oct 31 22:00:19.255957 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.255887    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb824101-1f12-403b-bf06-1051f6f25d17-maas-api-service-ca" pod "fb824101-1f12-403b-bf06-1051f6f25d17" (UID: "fb824101-1f12-403b-bf06-1051f6f25d17"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:00:19.256112 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.256088    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31d27730-27b7-4bfa-ac38-5a4a3cc13487-maas-api-service-ca" pod "31d27730-27b7-4bfa-ac38-5a4a3cc13487" (UID: "31d27730-27b7-4bfa-ac38-5a4a3cc13487"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:00:19.256671 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.256590    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/31d27730-27b7-4bfa-ac38-5a4a3cc13487-maas-api-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:19.256671 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.256617    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0864e283-2018-4c62-abeb-809f9dd12ce4-maas-api-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:19.256671 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.256633    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56cqw\" (UniqueName: \"kubernetes.io/projected/23b20e87-150e-48d7-9da9-d6c81de609d7-kube-api-access-56cqw\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:19.256671 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.256649    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb824101-1f12-403b-bf06-1051f6f25d17-maas-api-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:19.257819 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.257800    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb824101-1f12-403b-bf06-1051f6f25d17-kube-api-access-bbnc8" pod "fb824101-1f12-403b-bf06-1051f6f25d17" (UID: "fb824101-1f12-403b-bf06-1051f6f25d17"). InnerVolumeSpecName "kube-api-access-bbnc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:00:19.257888 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.257816    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0864e283-2018-4c62-abeb-809f9dd12ce4-kube-api-access-smjk8" pod "0864e283-2018-4c62-abeb-809f9dd12ce4" (UID: "0864e283-2018-4c62-abeb-809f9dd12ce4"). InnerVolumeSpecName "kube-api-access-smjk8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:00:19.257888 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.257828    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31d27730-27b7-4bfa-ac38-5a4a3cc13487-kube-api-access-gt9rp" pod "31d27730-27b7-4bfa-ac38-5a4a3cc13487" (UID: "31d27730-27b7-4bfa-ac38-5a4a3cc13487"). InnerVolumeSpecName "kube-api-access-gt9rp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:00:19.277028 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.276994    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f"]
Oct 31 22:00:19.283285 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.283254    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-cc45a7-29891400-kgj6f"]
Oct 31 22:00:19.358105 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.358023    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbnc8\" (UniqueName: \"kubernetes.io/projected/fb824101-1f12-403b-bf06-1051f6f25d17-kube-api-access-bbnc8\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:19.358105 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.358051    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smjk8\" (UniqueName: \"kubernetes.io/projected/0864e283-2018-4c62-abeb-809f9dd12ce4-kube-api-access-smjk8\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:19.358105 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.358062    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gt9rp\" (UniqueName: \"kubernetes.io/projected/31d27730-27b7-4bfa-ac38-5a4a3cc13487-kube-api-access-gt9rp\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:19.726714 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.726668    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k" podUID="2a792d40-6ac4-4859-ab09-71aceed3248f" containerName="payload-processing" containerID="cri-o://7ed8d98498db09c29374a79f06bcc1aac50045f76b3bf4c85d83a06b24c79462" gracePeriod=25
Oct 31 22:00:19.872601 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.872571    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk"
Oct 31 22:00:19.873063 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.872574    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk" event={"ID":"23b20e87-150e-48d7-9da9-d6c81de609d7","Type":"ContainerDied","Data":"7051a729dc682b981606cfaa2202815bb8d03658889d6bfcbb4075f3c3954206"}
Oct 31 22:00:19.873063 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.872722    4460 scope.go:122] "RemoveContainer" containerID="3657120c663bb820513b3d4382cd6845d72f962183a9319cf55aa212349bc012"
Oct 31 22:00:19.877665 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.876242    4460 generic.go:356] "Generic (PLEG): container finished" podID="0a11ccff-84e3-4179-9f19-f07b86783bd3" containerID="5fe99f96727506ccafbd1542c1a0b0473186408e9af80147ee066466583b3151" exitCode=60
Oct 31 22:00:19.877665 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.876409    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j" event={"ID":"0a11ccff-84e3-4179-9f19-f07b86783bd3","Type":"ContainerDied","Data":"5fe99f96727506ccafbd1542c1a0b0473186408e9af80147ee066466583b3151"}
Oct 31 22:00:19.877665 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.876833    4460 scope.go:122] "RemoveContainer" containerID="5fe99f96727506ccafbd1542c1a0b0473186408e9af80147ee066466583b3151"
Oct 31 22:00:19.877665 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:19.877210    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j_odh-ai-gateway-infra(0a11ccff-84e3-4179-9f19-f07b86783bd3)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j" podUID="0a11ccff-84e3-4179-9f19-f07b86783bd3"
Oct 31 22:00:19.884253 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.884227    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk"
Oct 31 22:00:19.884712 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.884688    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk" event={"ID":"31d27730-27b7-4bfa-ac38-5a4a3cc13487","Type":"ContainerDied","Data":"c48b80364022daf20c49d5127a0427f5ec68c3f5c4ee41139e01bd8a88ceae16"}
Oct 31 22:00:19.889123 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.889100    4460 scope.go:122] "RemoveContainer" containerID="d786bffa958f14b2a588a1b092a3ebb7832980548101598bd1695f4ccb5a50f0"
Oct 31 22:00:19.889647 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.889619    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w" event={"ID":"63aa3078-082d-400e-abfb-9985668831fc","Type":"ContainerStarted","Data":"eb55de2b214c4c1dbc9f4a48a7008655c210dc13184f8104275e38dee60d609e"}
Oct 31 22:00:19.893802 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.893780    4460 generic.go:356] "Generic (PLEG): container finished" podID="2a792d40-6ac4-4859-ab09-71aceed3248f" containerID="7ed8d98498db09c29374a79f06bcc1aac50045f76b3bf4c85d83a06b24c79462" exitCode=0
Oct 31 22:00:19.893911 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.893873    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k" event={"ID":"2a792d40-6ac4-4859-ab09-71aceed3248f","Type":"ContainerDied","Data":"7ed8d98498db09c29374a79f06bcc1aac50045f76b3bf4c85d83a06b24c79462"}
Oct 31 22:00:19.895901 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.895716    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n"
Oct 31 22:00:19.896289 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.895709    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n" event={"ID":"fb824101-1f12-403b-bf06-1051f6f25d17","Type":"ContainerDied","Data":"570728dceb250031f2c0d7a77b7fe37791a5efae709eaf6eec449a561ccd267a"}
Oct 31 22:00:19.898700 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.898675    4460 generic.go:356] "Generic (PLEG): container finished" podID="e081a034-3ce7-4af9-ba84-d44f463ae8fe" containerID="977628f4d7e340c525bdb2e0abfff1392d35431c2f9b53dd7779271fbdbec952" exitCode=60
Oct 31 22:00:19.898822 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.898806    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj" event={"ID":"e081a034-3ce7-4af9-ba84-d44f463ae8fe","Type":"ContainerDied","Data":"977628f4d7e340c525bdb2e0abfff1392d35431c2f9b53dd7779271fbdbec952"}
Oct 31 22:00:19.899261 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.899224    4460 scope.go:122] "RemoveContainer" containerID="977628f4d7e340c525bdb2e0abfff1392d35431c2f9b53dd7779271fbdbec952"
Oct 31 22:00:19.899567 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:19.899543    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj_odh-ai-gateway-infra(e081a034-3ce7-4af9-ba84-d44f463ae8fe)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj" podUID="e081a034-3ce7-4af9-ba84-d44f463ae8fe"
Oct 31 22:00:19.904501 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.903002    4460 generic.go:356] "Generic (PLEG): container finished" podID="093b9d9a-ec69-4552-a889-6f3bccda27cc" containerID="e1cd8cfd097f5953265e583ea38a965d1897a172c71de4fd72b795e0c53d0491" exitCode=60
Oct 31 22:00:19.904501 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.903125    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r" event={"ID":"093b9d9a-ec69-4552-a889-6f3bccda27cc","Type":"ContainerDied","Data":"e1cd8cfd097f5953265e583ea38a965d1897a172c71de4fd72b795e0c53d0491"}
Oct 31 22:00:19.904501 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.903551    4460 scope.go:122] "RemoveContainer" containerID="e1cd8cfd097f5953265e583ea38a965d1897a172c71de4fd72b795e0c53d0491"
Oct 31 22:00:19.904501 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:19.903870    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r_odh-ai-gateway-infra(093b9d9a-ec69-4552-a889-6f3bccda27cc)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r" podUID="093b9d9a-ec69-4552-a889-6f3bccda27cc"
Oct 31 22:00:19.905732 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.905707    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w" event={"ID":"0864e283-2018-4c62-abeb-809f9dd12ce4","Type":"ContainerDied","Data":"3ae701caacbdbc9274eeb9eac5c8530c87701018d2f65b2f14f552a3b77e6265"}
Oct 31 22:00:19.905824 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.905793    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w"
Oct 31 22:00:19.914170 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.914151    4460 scope.go:122] "RemoveContainer" containerID="55f1f02b10f75f191c86eee5d5cc7601c65f0fbb2a119ffaf625a218e4169718"
Oct 31 22:00:19.944696 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.944383    4460 scope.go:122] "RemoveContainer" containerID="520639a4bc156b9eb67d3e5fa66e2dc72900145d75be888a0abc1a3516820726"
Oct 31 22:00:19.958070 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.957907    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:19.961004 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.960982    4460 scope.go:122] "RemoveContainer" containerID="3b3f24b9b111142a25904c4b929c109dc8093dff991be52665aaf52b5c478fcf"
Oct 31 22:00:19.977389 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:19.977360    4460 scope.go:122] "RemoveContainer" containerID="c6a8f7e495c112c9344441d446700ae5ade9b9890e18f456edf6ea9b1453b2b4"
Oct 31 22:00:20.062675 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.062330    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk"]
Oct 31 22:00:20.064460 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.064433    4460 scope.go:122] "RemoveContainer" containerID="cdb4a08dfffa00604927c516fa792a0a9b6a4929270c79b8ff6ac951eb45747f"
Oct 31 22:00:20.069453 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.069398    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k"
Oct 31 22:00:20.072453 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.072410    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b36e62-29891400-qv2fk"]
Oct 31 22:00:20.109414 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.109362    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r"]
Oct 31 22:00:20.129933 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.129905    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq"
Oct 31 22:00:20.170985 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.170886    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/229deecb-4066-4ef7-bcf0-70bd5c15df45-kube-api-access-vp2f5\" (UniqueName: \"kubernetes.io/projected/229deecb-4066-4ef7-bcf0-70bd5c15df45-kube-api-access-vp2f5\") pod \"229deecb-4066-4ef7-bcf0-70bd5c15df45\" (UID: \"229deecb-4066-4ef7-bcf0-70bd5c15df45\") "
Oct 31 22:00:20.171141 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.171068    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2a792d40-6ac4-4859-ab09-71aceed3248f-provider-credentials\" (UniqueName: \"kubernetes.io/projected/2a792d40-6ac4-4859-ab09-71aceed3248f-provider-credentials\") pod \"2a792d40-6ac4-4859-ab09-71aceed3248f\" (UID: \"2a792d40-6ac4-4859-ab09-71aceed3248f\") "
Oct 31 22:00:20.171546 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.171520    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2a792d40-6ac4-4859-ab09-71aceed3248f-kube-api-access-nblsw\" (UniqueName: \"kubernetes.io/projected/2a792d40-6ac4-4859-ab09-71aceed3248f-kube-api-access-nblsw\") pod \"2a792d40-6ac4-4859-ab09-71aceed3248f\" (UID: \"2a792d40-6ac4-4859-ab09-71aceed3248f\") "
Oct 31 22:00:20.172528 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.172440    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/229deecb-4066-4ef7-bcf0-70bd5c15df45-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/229deecb-4066-4ef7-bcf0-70bd5c15df45-maas-api-service-ca\") pod \"229deecb-4066-4ef7-bcf0-70bd5c15df45\" (UID: \"229deecb-4066-4ef7-bcf0-70bd5c15df45\") "
Oct 31 22:00:20.172627 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.172609    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2a792d40-6ac4-4859-ab09-71aceed3248f-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2a792d40-6ac4-4859-ab09-71aceed3248f-routing-overlay-volume\") pod \"2a792d40-6ac4-4859-ab09-71aceed3248f\" (UID: \"2a792d40-6ac4-4859-ab09-71aceed3248f\") "
Oct 31 22:00:20.172719 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.172700    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2a792d40-6ac4-4859-ab09-71aceed3248f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a792d40-6ac4-4859-ab09-71aceed3248f-plugins-config-volume\") pod \"2a792d40-6ac4-4859-ab09-71aceed3248f\" (UID: \"2a792d40-6ac4-4859-ab09-71aceed3248f\") "
Oct 31 22:00:20.173184 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.173150    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a792d40-6ac4-4859-ab09-71aceed3248f-routing-overlay-volume" pod "2a792d40-6ac4-4859-ab09-71aceed3248f" (UID: "2a792d40-6ac4-4859-ab09-71aceed3248f"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:00:20.173616 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.173587    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a792d40-6ac4-4859-ab09-71aceed3248f-plugins-config-volume" pod "2a792d40-6ac4-4859-ab09-71aceed3248f" (UID: "2a792d40-6ac4-4859-ab09-71aceed3248f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:00:20.173718 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.173605    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/229deecb-4066-4ef7-bcf0-70bd5c15df45-maas-api-service-ca" pod "229deecb-4066-4ef7-bcf0-70bd5c15df45" (UID: "229deecb-4066-4ef7-bcf0-70bd5c15df45"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:00:20.174074 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.174050    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/229deecb-4066-4ef7-bcf0-70bd5c15df45-maas-api-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:20.174164 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.174076    4460 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/2a792d40-6ac4-4859-ab09-71aceed3248f-routing-overlay-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:20.174164 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.174094    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a792d40-6ac4-4859-ab09-71aceed3248f-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:20.174762 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.174738    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a792d40-6ac4-4859-ab09-71aceed3248f-kube-api-access-nblsw" pod "2a792d40-6ac4-4859-ab09-71aceed3248f" (UID: "2a792d40-6ac4-4859-ab09-71aceed3248f"). InnerVolumeSpecName "kube-api-access-nblsw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:00:20.174841 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.174821    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a792d40-6ac4-4859-ab09-71aceed3248f-provider-credentials" pod "2a792d40-6ac4-4859-ab09-71aceed3248f" (UID: "2a792d40-6ac4-4859-ab09-71aceed3248f"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:00:20.175826 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.175805    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/229deecb-4066-4ef7-bcf0-70bd5c15df45-kube-api-access-vp2f5" pod "229deecb-4066-4ef7-bcf0-70bd5c15df45" (UID: "229deecb-4066-4ef7-bcf0-70bd5c15df45"). InnerVolumeSpecName "kube-api-access-vp2f5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:00:20.204023 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.203993    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj"]
Oct 31 22:00:20.209744 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.209721    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n"]
Oct 31 22:00:20.229868 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.229842    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="23b20e87-150e-48d7-9da9-d6c81de609d7" path="/var/lib/kubelet/pods/23b20e87-150e-48d7-9da9-d6c81de609d7/volumes"
Oct 31 22:00:20.230407 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.230387    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="76255444-0212-46b9-9a6d-571bc0ba8f28" path="/var/lib/kubelet/pods/76255444-0212-46b9-9a6d-571bc0ba8f28/volumes"
Oct 31 22:00:20.231146 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.231130    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-56a6d4-29891400-4k48n"]
Oct 31 22:00:20.248706 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.248686    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j"]
Oct 31 22:00:20.276059 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.275977    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vp2f5\" (UniqueName: \"kubernetes.io/projected/229deecb-4066-4ef7-bcf0-70bd5c15df45-kube-api-access-vp2f5\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:20.276059 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.276014    4460 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/2a792d40-6ac4-4859-ab09-71aceed3248f-provider-credentials\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:20.276059 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.276033    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nblsw\" (UniqueName: \"kubernetes.io/projected/2a792d40-6ac4-4859-ab09-71aceed3248f-kube-api-access-nblsw\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:20.304898 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.304865    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w"]
Oct 31 22:00:20.309064 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.309033    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-2dbae9-29891400-rv94w"]
Oct 31 22:00:20.348302 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.348272    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk"]
Oct 31 22:00:20.354260 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.354230    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w0-5237cd-29891400-8jksk"]
Oct 31 22:00:20.923367 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.923338    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq"
Oct 31 22:00:20.923367 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.923347    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq" event={"ID":"229deecb-4066-4ef7-bcf0-70bd5c15df45","Type":"ContainerDied","Data":"d5928258fd07d86b318ebef3cf596a1859a64a6440283296611dbc97096dcc6e"}
Oct 31 22:00:20.923790 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.923456    4460 scope.go:122] "RemoveContainer" containerID="55d2c879387bde7319cddfd037e9f503975dad200ecbf1699b9e3ed945fa626e"
Oct 31 22:00:20.927427 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.927393    4460 generic.go:356] "Generic (PLEG): container finished" podID="63aa3078-082d-400e-abfb-9985668831fc" containerID="eb55de2b214c4c1dbc9f4a48a7008655c210dc13184f8104275e38dee60d609e" exitCode=60
Oct 31 22:00:20.927531 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.927512    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w" event={"ID":"63aa3078-082d-400e-abfb-9985668831fc","Type":"ContainerDied","Data":"eb55de2b214c4c1dbc9f4a48a7008655c210dc13184f8104275e38dee60d609e"}
Oct 31 22:00:20.928020 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.928000    4460 scope.go:122] "RemoveContainer" containerID="eb55de2b214c4c1dbc9f4a48a7008655c210dc13184f8104275e38dee60d609e"
Oct 31 22:00:20.928411 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:20.928387    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w_odh-ai-gateway-infra(63aa3078-082d-400e-abfb-9985668831fc)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w" podUID="63aa3078-082d-400e-abfb-9985668831fc"
Oct 31 22:00:20.930532 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.930509    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k" event={"ID":"2a792d40-6ac4-4859-ab09-71aceed3248f","Type":"ContainerDied","Data":"c836a7210ca292763676396daacbe358acd550f1ee4bfc3c1c0e08e381b62353"}
Oct 31 22:00:20.930693 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.930537    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k"
Oct 31 22:00:20.938063 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.938042    4460 scope.go:122] "RemoveContainer" containerID="899d86b8923e4c31b1c37e65e619f7c2170aaa2a6ff934bb0b7f852ad35674e2"
Oct 31 22:00:20.964260 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.963559    4460 scope.go:122] "RemoveContainer" containerID="7ed8d98498db09c29374a79f06bcc1aac50045f76b3bf4c85d83a06b24c79462"
Oct 31 22:00:20.972766 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:20.972501    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w"]
Oct 31 22:00:21.051992 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.051957    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k"]
Oct 31 22:00:21.058541 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.058518    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-5bf6fqt4k"]
Oct 31 22:00:21.143221 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.143190    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj"
Oct 31 22:00:21.154487 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.154448    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq"]
Oct 31 22:00:21.161808 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.161251    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-4bd51b-29891400-2r4vq"]
Oct 31 22:00:21.197633 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.197608    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j"
Oct 31 22:00:21.205256 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.205229    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e081a034-3ce7-4af9-ba84-d44f463ae8fe-kube-api-access-cv6b2\" (UniqueName: \"kubernetes.io/projected/e081a034-3ce7-4af9-ba84-d44f463ae8fe-kube-api-access-cv6b2\") pod \"e081a034-3ce7-4af9-ba84-d44f463ae8fe\" (UID: \"e081a034-3ce7-4af9-ba84-d44f463ae8fe\") "
Oct 31 22:00:21.205416 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.205363    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e081a034-3ce7-4af9-ba84-d44f463ae8fe-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e081a034-3ce7-4af9-ba84-d44f463ae8fe-maas-api-service-ca\") pod \"e081a034-3ce7-4af9-ba84-d44f463ae8fe\" (UID: \"e081a034-3ce7-4af9-ba84-d44f463ae8fe\") "
Oct 31 22:00:21.205877 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.205847    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e081a034-3ce7-4af9-ba84-d44f463ae8fe-maas-api-service-ca" pod "e081a034-3ce7-4af9-ba84-d44f463ae8fe" (UID: "e081a034-3ce7-4af9-ba84-d44f463ae8fe"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:00:21.206231 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.206210    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e081a034-3ce7-4af9-ba84-d44f463ae8fe-maas-api-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:21.208216 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.208194    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r"
Oct 31 22:00:21.209047 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.209026    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e081a034-3ce7-4af9-ba84-d44f463ae8fe-kube-api-access-cv6b2" pod "e081a034-3ce7-4af9-ba84-d44f463ae8fe" (UID: "e081a034-3ce7-4af9-ba84-d44f463ae8fe"). InnerVolumeSpecName "kube-api-access-cv6b2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:00:21.307056 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.307022    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0a11ccff-84e3-4179-9f19-f07b86783bd3-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a11ccff-84e3-4179-9f19-f07b86783bd3-maas-api-service-ca\") pod \"0a11ccff-84e3-4179-9f19-f07b86783bd3\" (UID: \"0a11ccff-84e3-4179-9f19-f07b86783bd3\") "
Oct 31 22:00:21.307208 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.307092    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/093b9d9a-ec69-4552-a889-6f3bccda27cc-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/093b9d9a-ec69-4552-a889-6f3bccda27cc-maas-api-service-ca\") pod \"093b9d9a-ec69-4552-a889-6f3bccda27cc\" (UID: \"093b9d9a-ec69-4552-a889-6f3bccda27cc\") "
Oct 31 22:00:21.307294 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.307276    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/093b9d9a-ec69-4552-a889-6f3bccda27cc-kube-api-access-9gntm\" (UniqueName: \"kubernetes.io/projected/093b9d9a-ec69-4552-a889-6f3bccda27cc-kube-api-access-9gntm\") pod \"093b9d9a-ec69-4552-a889-6f3bccda27cc\" (UID: \"093b9d9a-ec69-4552-a889-6f3bccda27cc\") "
Oct 31 22:00:21.307404 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.307380    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a11ccff-84e3-4179-9f19-f07b86783bd3-kube-api-access-ghw5b\" (UniqueName: \"kubernetes.io/projected/0a11ccff-84e3-4179-9f19-f07b86783bd3-kube-api-access-ghw5b\") pod \"0a11ccff-84e3-4179-9f19-f07b86783bd3\" (UID: \"0a11ccff-84e3-4179-9f19-f07b86783bd3\") "
Oct 31 22:00:21.307673 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.307647    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a11ccff-84e3-4179-9f19-f07b86783bd3-maas-api-service-ca" pod "0a11ccff-84e3-4179-9f19-f07b86783bd3" (UID: "0a11ccff-84e3-4179-9f19-f07b86783bd3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:00:21.308140 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.308116    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cv6b2\" (UniqueName: \"kubernetes.io/projected/e081a034-3ce7-4af9-ba84-d44f463ae8fe-kube-api-access-cv6b2\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:21.308224 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.308145    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a11ccff-84e3-4179-9f19-f07b86783bd3-maas-api-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:21.308869 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.308845    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/093b9d9a-ec69-4552-a889-6f3bccda27cc-maas-api-service-ca" pod "093b9d9a-ec69-4552-a889-6f3bccda27cc" (UID: "093b9d9a-ec69-4552-a889-6f3bccda27cc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:00:21.316975 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.310373    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a11ccff-84e3-4179-9f19-f07b86783bd3-kube-api-access-ghw5b" pod "0a11ccff-84e3-4179-9f19-f07b86783bd3" (UID: "0a11ccff-84e3-4179-9f19-f07b86783bd3"). InnerVolumeSpecName "kube-api-access-ghw5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:00:21.316975 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.311147    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/093b9d9a-ec69-4552-a889-6f3bccda27cc-kube-api-access-9gntm" pod "093b9d9a-ec69-4552-a889-6f3bccda27cc" (UID: "093b9d9a-ec69-4552-a889-6f3bccda27cc"). InnerVolumeSpecName "kube-api-access-9gntm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:00:21.409621 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.409577    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9gntm\" (UniqueName: \"kubernetes.io/projected/093b9d9a-ec69-4552-a889-6f3bccda27cc-kube-api-access-9gntm\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:21.409621 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.409613    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghw5b\" (UniqueName: \"kubernetes.io/projected/0a11ccff-84e3-4179-9f19-f07b86783bd3-kube-api-access-ghw5b\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:21.409800 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.409629    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/093b9d9a-ec69-4552-a889-6f3bccda27cc-maas-api-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:21.457792 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:21.457673    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:00:21.910972 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.910909    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:21.939822 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.939795    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r"
Oct 31 22:00:21.940275 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.939835    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r" event={"ID":"093b9d9a-ec69-4552-a889-6f3bccda27cc","Type":"ContainerDied","Data":"dbbdfcc9c466c875778d9ac469127cd59eb65520a96f8385bbc5237beb6039ec"}
Oct 31 22:00:21.940275 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.939910    4460 scope.go:122] "RemoveContainer" containerID="e1cd8cfd097f5953265e583ea38a965d1897a172c71de4fd72b795e0c53d0491"
Oct 31 22:00:21.941646 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.941620    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j"
Oct 31 22:00:21.941865 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.941839    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j" event={"ID":"0a11ccff-84e3-4179-9f19-f07b86783bd3","Type":"ContainerDied","Data":"6a64122fc869a80fa3000cac008c0f744419f6324ee706ca762d63b11a8398c5"}
Oct 31 22:00:21.947484 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.947461    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj" event={"ID":"e081a034-3ce7-4af9-ba84-d44f463ae8fe","Type":"ContainerDied","Data":"ed1ffd915c9ec2934c0c3b1e597f12ac950743b6b646f98ee1cafe0f4277adb6"}
Oct 31 22:00:21.947575 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.947488    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj"
Oct 31 22:00:21.988686 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:21.982802    4460 scope.go:122] "RemoveContainer" containerID="5fe99f96727506ccafbd1542c1a0b0473186408e9af80147ee066466583b3151"
Oct 31 22:00:22.009062 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.009016    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r"]
Oct 31 22:00:22.011488 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.011458    4460 scope.go:122] "RemoveContainer" containerID="977628f4d7e340c525bdb2e0abfff1392d35431c2f9b53dd7779271fbdbec952"
Oct 31 22:00:22.011911 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.011837    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-f1e791-29891400-ghn7r"]
Oct 31 22:00:22.109243 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.109187    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j"]
Oct 31 22:00:22.113805 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.113779    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-fac043-29891400-2fk8j"]
Oct 31 22:00:22.151363 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.151336    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj"]
Oct 31 22:00:22.155435 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.155412    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w"
Oct 31 22:00:22.156266 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.156244    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-c16684-29891400-9bbcj"]
Oct 31 22:00:22.224774 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.224686    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/63aa3078-082d-400e-abfb-9985668831fc-kube-api-access-wxqbm\" (UniqueName: \"kubernetes.io/projected/63aa3078-082d-400e-abfb-9985668831fc-kube-api-access-wxqbm\") pod \"63aa3078-082d-400e-abfb-9985668831fc\" (UID: \"63aa3078-082d-400e-abfb-9985668831fc\") "
Oct 31 22:00:22.224960 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.224886    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/63aa3078-082d-400e-abfb-9985668831fc-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63aa3078-082d-400e-abfb-9985668831fc-maas-api-service-ca\") pod \"63aa3078-082d-400e-abfb-9985668831fc\" (UID: \"63aa3078-082d-400e-abfb-9985668831fc\") "
Oct 31 22:00:22.225753 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.225718    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63aa3078-082d-400e-abfb-9985668831fc-maas-api-service-ca" pod "63aa3078-082d-400e-abfb-9985668831fc" (UID: "63aa3078-082d-400e-abfb-9985668831fc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:00:22.227661 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.227627    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63aa3078-082d-400e-abfb-9985668831fc-kube-api-access-wxqbm" pod "63aa3078-082d-400e-abfb-9985668831fc" (UID: "63aa3078-082d-400e-abfb-9985668831fc"). InnerVolumeSpecName "kube-api-access-wxqbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:00:22.233411 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.233382    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0864e283-2018-4c62-abeb-809f9dd12ce4" path="/var/lib/kubelet/pods/0864e283-2018-4c62-abeb-809f9dd12ce4/volumes"
Oct 31 22:00:22.233881 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.233862    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="093b9d9a-ec69-4552-a889-6f3bccda27cc" path="/var/lib/kubelet/pods/093b9d9a-ec69-4552-a889-6f3bccda27cc/volumes"
Oct 31 22:00:22.234322 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.234305    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0a11ccff-84e3-4179-9f19-f07b86783bd3" path="/var/lib/kubelet/pods/0a11ccff-84e3-4179-9f19-f07b86783bd3/volumes"
Oct 31 22:00:22.235085 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.235065    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="229deecb-4066-4ef7-bcf0-70bd5c15df45" path="/var/lib/kubelet/pods/229deecb-4066-4ef7-bcf0-70bd5c15df45/volumes"
Oct 31 22:00:22.235620 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.235592    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2a792d40-6ac4-4859-ab09-71aceed3248f" path="/var/lib/kubelet/pods/2a792d40-6ac4-4859-ab09-71aceed3248f/volumes"
Oct 31 22:00:22.236087 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.236068    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="31d27730-27b7-4bfa-ac38-5a4a3cc13487" path="/var/lib/kubelet/pods/31d27730-27b7-4bfa-ac38-5a4a3cc13487/volumes"
Oct 31 22:00:22.236992 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.236924    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e081a034-3ce7-4af9-ba84-d44f463ae8fe" path="/var/lib/kubelet/pods/e081a034-3ce7-4af9-ba84-d44f463ae8fe/volumes"
Oct 31 22:00:22.237603 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.237583    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fb824101-1f12-403b-bf06-1051f6f25d17" path="/var/lib/kubelet/pods/fb824101-1f12-403b-bf06-1051f6f25d17/volumes"
Oct 31 22:00:22.323036 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.322996    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:22.326710 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.326685    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxqbm\" (UniqueName: \"kubernetes.io/projected/63aa3078-082d-400e-abfb-9985668831fc-kube-api-access-wxqbm\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:22.326833 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.326712    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63aa3078-082d-400e-abfb-9985668831fc-maas-api-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:22.468974 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.451506    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:22.768261 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.768225    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8"]
Oct 31 22:00:22.771973 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.771929    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8"
Oct 31 22:00:22.805546 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.805520    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm"]
Oct 31 22:00:22.806250 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806235    4460 cpu_manager.go:414] "containerMap: removing container" podUID="e081a034-3ce7-4af9-ba84-d44f463ae8fe" containerName="cleanup"
Oct 31 22:00:22.806329 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806252    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e081a034-3ce7-4af9-ba84-d44f463ae8fe" containerName="cleanup"
Oct 31 22:00:22.806329 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806264    4460 cpu_manager.go:414] "containerMap: removing container" podUID="31d27730-27b7-4bfa-ac38-5a4a3cc13487" containerName="cleanup"
Oct 31 22:00:22.806329 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806269    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="31d27730-27b7-4bfa-ac38-5a4a3cc13487" containerName="cleanup"
Oct 31 22:00:22.806329 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806278    4460 cpu_manager.go:414] "containerMap: removing container" podUID="63aa3078-082d-400e-abfb-9985668831fc" containerName="cleanup"
Oct 31 22:00:22.806329 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806283    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="63aa3078-082d-400e-abfb-9985668831fc" containerName="cleanup"
Oct 31 22:00:22.806329 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806290    4460 cpu_manager.go:414] "containerMap: removing container" podUID="fb824101-1f12-403b-bf06-1051f6f25d17" containerName="cleanup"
Oct 31 22:00:22.806329 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806295    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fb824101-1f12-403b-bf06-1051f6f25d17" containerName="cleanup"
Oct 31 22:00:22.806329 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806302    4460 cpu_manager.go:414] "containerMap: removing container" podUID="fb824101-1f12-403b-bf06-1051f6f25d17" containerName="cleanup"
Oct 31 22:00:22.806329 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806307    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fb824101-1f12-403b-bf06-1051f6f25d17" containerName="cleanup"
Oct 31 22:00:22.806329 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806317    4460 cpu_manager.go:414] "containerMap: removing container" podUID="0a11ccff-84e3-4179-9f19-f07b86783bd3" containerName="cleanup"
Oct 31 22:00:22.806329 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806321    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a11ccff-84e3-4179-9f19-f07b86783bd3" containerName="cleanup"
Oct 31 22:00:22.806329 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806329    4460 cpu_manager.go:414] "containerMap: removing container" podUID="0864e283-2018-4c62-abeb-809f9dd12ce4" containerName="cleanup"
Oct 31 22:00:22.806329 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806334    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0864e283-2018-4c62-abeb-809f9dd12ce4" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806341    4460 cpu_manager.go:414] "containerMap: removing container" podUID="23b20e87-150e-48d7-9da9-d6c81de609d7" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806345    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="23b20e87-150e-48d7-9da9-d6c81de609d7" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806358    4460 cpu_manager.go:414] "containerMap: removing container" podUID="0a11ccff-84e3-4179-9f19-f07b86783bd3" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806363    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a11ccff-84e3-4179-9f19-f07b86783bd3" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806369    4460 cpu_manager.go:414] "containerMap: removing container" podUID="31d27730-27b7-4bfa-ac38-5a4a3cc13487" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806373    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="31d27730-27b7-4bfa-ac38-5a4a3cc13487" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806380    4460 cpu_manager.go:414] "containerMap: removing container" podUID="63aa3078-082d-400e-abfb-9985668831fc" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806384    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="63aa3078-082d-400e-abfb-9985668831fc" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806391    4460 cpu_manager.go:414] "containerMap: removing container" podUID="093b9d9a-ec69-4552-a889-6f3bccda27cc" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806395    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="093b9d9a-ec69-4552-a889-6f3bccda27cc" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806405    4460 cpu_manager.go:414] "containerMap: removing container" podUID="23b20e87-150e-48d7-9da9-d6c81de609d7" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806409    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="23b20e87-150e-48d7-9da9-d6c81de609d7" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806420    4460 cpu_manager.go:414] "containerMap: removing container" podUID="e081a034-3ce7-4af9-ba84-d44f463ae8fe" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806424    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e081a034-3ce7-4af9-ba84-d44f463ae8fe" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806430    4460 cpu_manager.go:414] "containerMap: removing container" podUID="0864e283-2018-4c62-abeb-809f9dd12ce4" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806434    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0864e283-2018-4c62-abeb-809f9dd12ce4" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806439    4460 cpu_manager.go:414] "containerMap: removing container" podUID="229deecb-4066-4ef7-bcf0-70bd5c15df45" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806444    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="229deecb-4066-4ef7-bcf0-70bd5c15df45" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806451    4460 cpu_manager.go:414] "containerMap: removing container" podUID="76255444-0212-46b9-9a6d-571bc0ba8f28" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806456    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="76255444-0212-46b9-9a6d-571bc0ba8f28" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806462    4460 cpu_manager.go:414] "containerMap: removing container" podUID="76255444-0212-46b9-9a6d-571bc0ba8f28" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806467    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="76255444-0212-46b9-9a6d-571bc0ba8f28" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806474    4460 cpu_manager.go:414] "containerMap: removing container" podUID="229deecb-4066-4ef7-bcf0-70bd5c15df45" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806480    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="229deecb-4066-4ef7-bcf0-70bd5c15df45" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806487    4460 cpu_manager.go:414] "containerMap: removing container" podUID="2a792d40-6ac4-4859-ab09-71aceed3248f" containerName="payload-processing"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806491    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2a792d40-6ac4-4859-ab09-71aceed3248f" containerName="payload-processing"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806507    4460 cpu_manager.go:414] "containerMap: removing container" podUID="093b9d9a-ec69-4552-a889-6f3bccda27cc" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806514    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="093b9d9a-ec69-4552-a889-6f3bccda27cc" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806622    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="63aa3078-082d-400e-abfb-9985668831fc" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806630    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="31d27730-27b7-4bfa-ac38-5a4a3cc13487" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806637    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="0864e283-2018-4c62-abeb-809f9dd12ce4" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806643    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="229deecb-4066-4ef7-bcf0-70bd5c15df45" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806648    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="23b20e87-150e-48d7-9da9-d6c81de609d7" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806656    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="e081a034-3ce7-4af9-ba84-d44f463ae8fe" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806663    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="31d27730-27b7-4bfa-ac38-5a4a3cc13487" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806670    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="63aa3078-082d-400e-abfb-9985668831fc" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806677    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="fb824101-1f12-403b-bf06-1051f6f25d17" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806682    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="fb824101-1f12-403b-bf06-1051f6f25d17" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806688    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a11ccff-84e3-4179-9f19-f07b86783bd3" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806694    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="23b20e87-150e-48d7-9da9-d6c81de609d7" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806701    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a11ccff-84e3-4179-9f19-f07b86783bd3" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806707    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="63aa3078-082d-400e-abfb-9985668831fc" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806712    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="093b9d9a-ec69-4552-a889-6f3bccda27cc" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806720    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="e081a034-3ce7-4af9-ba84-d44f463ae8fe" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806726    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="e081a034-3ce7-4af9-ba84-d44f463ae8fe" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806732    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="093b9d9a-ec69-4552-a889-6f3bccda27cc" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806737    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="0864e283-2018-4c62-abeb-809f9dd12ce4" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806744    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="76255444-0212-46b9-9a6d-571bc0ba8f28" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806749    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="76255444-0212-46b9-9a6d-571bc0ba8f28" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806753    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="229deecb-4066-4ef7-bcf0-70bd5c15df45" containerName="cleanup"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806759    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="2a792d40-6ac4-4859-ab09-71aceed3248f" containerName="payload-processing"
Oct 31 22:00:22.806779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.806764    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="76255444-0212-46b9-9a6d-571bc0ba8f28" containerName="cleanup"
Oct 31 22:00:22.811857 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.811840    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm"
Oct 31 22:00:22.829126 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.829061    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm"]
Oct 31 22:00:22.958078 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.958047    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w"
Oct 31 22:00:22.958078 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.958068    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w" event={"ID":"63aa3078-082d-400e-abfb-9985668831fc","Type":"ContainerDied","Data":"f1ae326067255a89263e3d75cba17141ed94db7a2aeda6764457a3bfee63e55b"}
Oct 31 22:00:22.958589 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.958130    4460 scope.go:122] "RemoveContainer" containerID="eb55de2b214c4c1dbc9f4a48a7008655c210dc13184f8104275e38dee60d609e"
Oct 31 22:00:22.962372 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.962303    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/81c4b66d-2e79-487c-80cd-69a66d76a3bf-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm\" (UID: \"81c4b66d-2e79-487c-80cd-69a66d76a3bf\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm"
Oct 31 22:00:22.962523 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.962419    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81c4b66d-2e79-487c-80cd-69a66d76a3bf-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm\" (UID: \"81c4b66d-2e79-487c-80cd-69a66d76a3bf\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm"
Oct 31 22:00:22.962625 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.962563    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qvqb\" (UniqueName: \"kubernetes.io/projected/81c4b66d-2e79-487c-80cd-69a66d76a3bf-kube-api-access-8qvqb\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm\" (UID: \"81c4b66d-2e79-487c-80cd-69a66d76a3bf\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm"
Oct 31 22:00:22.962625 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:22.962595    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/81c4b66d-2e79-487c-80cd-69a66d76a3bf-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm\" (UID: \"81c4b66d-2e79-487c-80cd-69a66d76a3bf\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm"
Oct 31 22:00:23.115828 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:23.115381    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qvqb\" (UniqueName: \"kubernetes.io/projected/81c4b66d-2e79-487c-80cd-69a66d76a3bf-kube-api-access-8qvqb\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm\" (UID: \"81c4b66d-2e79-487c-80cd-69a66d76a3bf\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm"
Oct 31 22:00:23.115828 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:23.115460    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/81c4b66d-2e79-487c-80cd-69a66d76a3bf-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm\" (UID: \"81c4b66d-2e79-487c-80cd-69a66d76a3bf\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm"
Oct 31 22:00:23.115828 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:23.115613    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/81c4b66d-2e79-487c-80cd-69a66d76a3bf-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm\" (UID: \"81c4b66d-2e79-487c-80cd-69a66d76a3bf\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm"
Oct 31 22:00:23.115828 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:23.115693    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81c4b66d-2e79-487c-80cd-69a66d76a3bf-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm\" (UID: \"81c4b66d-2e79-487c-80cd-69a66d76a3bf\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm"
Oct 31 22:00:23.135966 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:23.116731    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81c4b66d-2e79-487c-80cd-69a66d76a3bf-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm\" (UID: \"81c4b66d-2e79-487c-80cd-69a66d76a3bf\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm"
Oct 31 22:00:23.139187 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:23.139142    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/81c4b66d-2e79-487c-80cd-69a66d76a3bf-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm\" (UID: \"81c4b66d-2e79-487c-80cd-69a66d76a3bf\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm"
Oct 31 22:00:23.143099 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:23.143068    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w"]
Oct 31 22:00:23.143244 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:23.143123    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-c744c0-29891400-85g4w"]
Oct 31 22:00:23.150045 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:23.150016    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/81c4b66d-2e79-487c-80cd-69a66d76a3bf-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm\" (UID: \"81c4b66d-2e79-487c-80cd-69a66d76a3bf\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm"
Oct 31 22:00:23.167532 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:23.167472    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qvqb\" (UniqueName: \"kubernetes.io/projected/81c4b66d-2e79-487c-80cd-69a66d76a3bf-kube-api-access-8qvqb\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm\" (UID: \"81c4b66d-2e79-487c-80cd-69a66d76a3bf\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm"
Oct 31 22:00:23.429672 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:23.429595    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm"
Oct 31 22:00:23.732043 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:23.731960    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:23.740858 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:23.740827    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm"]
Oct 31 22:00:23.741598 ip-10-0-136-157 kubenswrapper[4460]: W1031 22:00:23.741571    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81c4b66d_2e79_487c_80cd_69a66d76a3bf.slice/crio-dd5227d01a494a36e1cc776d23f35d5398a12214dbdcb175918634b5d48729a2 WatchSource:0}: Error finding container dd5227d01a494a36e1cc776d23f35d5398a12214dbdcb175918634b5d48729a2: Status 404 returned error can't find the container with id dd5227d01a494a36e1cc776d23f35d5398a12214dbdcb175918634b5d48729a2
Oct 31 22:00:23.971400 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:23.971357    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm" event={"ID":"81c4b66d-2e79-487c-80cd-69a66d76a3bf","Type":"ContainerStarted","Data":"c28721fec3b2e4bcbaf37f8e3d681964daefbe65d8d286dd44362ea8dbada12d"}
Oct 31 22:00:23.971861 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:23.971428    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm" event={"ID":"81c4b66d-2e79-487c-80cd-69a66d76a3bf","Type":"ContainerStarted","Data":"dd5227d01a494a36e1cc776d23f35d5398a12214dbdcb175918634b5d48729a2"}
Oct 31 22:00:23.971861 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:23.971572    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm"
Oct 31 22:00:24.017712 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:24.017662    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm" podStartSLOduration=2.017648842 podStartE2EDuration="2.017648842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:00:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:00:24.015474749 +0000 UTC m=+2663.931784179" watchObservedRunningTime="2026-10-31 22:00:24.017648842 +0000 UTC m=+2663.933958272"
Oct 31 22:00:24.227161 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:24.227132    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="63aa3078-082d-400e-abfb-9985668831fc" path="/var/lib/kubelet/pods/63aa3078-082d-400e-abfb-9985668831fc/volumes"
Oct 31 22:00:25.470866 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:25.470830    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:27.179908 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:27.179872    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:27.768595 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:27.768544    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8" podUID="12241bee-ccf4-4a34-8e1a-cfec063d543e" containerName="payload-processing" containerID="cri-o://6a587722563da71d6dbd424faf05b868f905be0e70b8dbd2530ad8443c770bef" gracePeriod=25
Oct 31 22:00:28.005584 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:28.005540    4460 generic.go:356] "Generic (PLEG): container finished" podID="12241bee-ccf4-4a34-8e1a-cfec063d543e" containerID="6a587722563da71d6dbd424faf05b868f905be0e70b8dbd2530ad8443c770bef" exitCode=0
Oct 31 22:00:28.005759 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:28.005611    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8" event={"ID":"12241bee-ccf4-4a34-8e1a-cfec063d543e","Type":"ContainerDied","Data":"6a587722563da71d6dbd424faf05b868f905be0e70b8dbd2530ad8443c770bef"}
Oct 31 22:00:28.068473 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:28.068444    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8"
Oct 31 22:00:28.091381 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:28.091354    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/12241bee-ccf4-4a34-8e1a-cfec063d543e-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/12241bee-ccf4-4a34-8e1a-cfec063d543e-routing-overlay-volume\") pod \"12241bee-ccf4-4a34-8e1a-cfec063d543e\" (UID: \"12241bee-ccf4-4a34-8e1a-cfec063d543e\") "
Oct 31 22:00:28.091508 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:28.091399    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/12241bee-ccf4-4a34-8e1a-cfec063d543e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12241bee-ccf4-4a34-8e1a-cfec063d543e-plugins-config-volume\") pod \"12241bee-ccf4-4a34-8e1a-cfec063d543e\" (UID: \"12241bee-ccf4-4a34-8e1a-cfec063d543e\") "
Oct 31 22:00:28.091508 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:28.091483    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/12241bee-ccf4-4a34-8e1a-cfec063d543e-provider-credentials\" (UniqueName: \"kubernetes.io/projected/12241bee-ccf4-4a34-8e1a-cfec063d543e-provider-credentials\") pod \"12241bee-ccf4-4a34-8e1a-cfec063d543e\" (UID: \"12241bee-ccf4-4a34-8e1a-cfec063d543e\") "
Oct 31 22:00:28.091677 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:28.091659    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/12241bee-ccf4-4a34-8e1a-cfec063d543e-kube-api-access-dxnpd\" (UniqueName: \"kubernetes.io/projected/12241bee-ccf4-4a34-8e1a-cfec063d543e-kube-api-access-dxnpd\") pod \"12241bee-ccf4-4a34-8e1a-cfec063d543e\" (UID: \"12241bee-ccf4-4a34-8e1a-cfec063d543e\") "
Oct 31 22:00:28.091738 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:28.091715    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12241bee-ccf4-4a34-8e1a-cfec063d543e-routing-overlay-volume" pod "12241bee-ccf4-4a34-8e1a-cfec063d543e" (UID: "12241bee-ccf4-4a34-8e1a-cfec063d543e"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:00:28.091895 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:28.091873    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12241bee-ccf4-4a34-8e1a-cfec063d543e-plugins-config-volume" pod "12241bee-ccf4-4a34-8e1a-cfec063d543e" (UID: "12241bee-ccf4-4a34-8e1a-cfec063d543e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:00:28.092243 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:28.092221    4460 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/12241bee-ccf4-4a34-8e1a-cfec063d543e-routing-overlay-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:28.092243 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:28.092244    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12241bee-ccf4-4a34-8e1a-cfec063d543e-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:28.093807 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:28.093786    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12241bee-ccf4-4a34-8e1a-cfec063d543e-provider-credentials" pod "12241bee-ccf4-4a34-8e1a-cfec063d543e" (UID: "12241bee-ccf4-4a34-8e1a-cfec063d543e"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:00:28.093904 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:28.093820    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12241bee-ccf4-4a34-8e1a-cfec063d543e-kube-api-access-dxnpd" pod "12241bee-ccf4-4a34-8e1a-cfec063d543e" (UID: "12241bee-ccf4-4a34-8e1a-cfec063d543e"). InnerVolumeSpecName "kube-api-access-dxnpd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:00:28.193683 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:28.193640    4460 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/12241bee-ccf4-4a34-8e1a-cfec063d543e-provider-credentials\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:28.193683 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:28.193674    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxnpd\" (UniqueName: \"kubernetes.io/projected/12241bee-ccf4-4a34-8e1a-cfec063d543e-kube-api-access-dxnpd\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:28.950429 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:28.950391    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:29.028785 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:29.028719    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8" event={"ID":"12241bee-ccf4-4a34-8e1a-cfec063d543e","Type":"ContainerDied","Data":"faec8dbb4130776d4b75b70a23eb79c5839a1d6d3df55971d9b36dec673d64f2"}
Oct 31 22:00:29.029046 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:29.028805    4460 scope.go:122] "RemoveContainer" containerID="6a587722563da71d6dbd424faf05b868f905be0e70b8dbd2530ad8443c770bef"
Oct 31 22:00:29.029046 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:29.029032    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8"
Oct 31 22:00:29.092449 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:29.092417    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8"]
Oct 31 22:00:29.104673 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:29.104644    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-5ccchzbc8"]
Oct 31 22:00:30.229513 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:30.229475    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="12241bee-ccf4-4a34-8e1a-cfec063d543e" path="/var/lib/kubelet/pods/12241bee-ccf4-4a34-8e1a-cfec063d543e/volumes"
Oct 31 22:00:30.893694 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:30.893648    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:32.099839 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.099782    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:32.212593 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.212542    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:32.818794 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.818762    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:32.901213 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.901176    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p"]
Oct 31 22:00:32.902220 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.902193    4460 cpu_manager.go:414] "containerMap: removing container" podUID="e081a034-3ce7-4af9-ba84-d44f463ae8fe" containerName="cleanup"
Oct 31 22:00:32.902220 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.902218    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e081a034-3ce7-4af9-ba84-d44f463ae8fe" containerName="cleanup"
Oct 31 22:00:32.902459 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.902238    4460 cpu_manager.go:414] "containerMap: removing container" podUID="31d27730-27b7-4bfa-ac38-5a4a3cc13487" containerName="cleanup"
Oct 31 22:00:32.902459 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.902246    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="31d27730-27b7-4bfa-ac38-5a4a3cc13487" containerName="cleanup"
Oct 31 22:00:32.902459 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.902256    4460 cpu_manager.go:414] "containerMap: removing container" podUID="63aa3078-082d-400e-abfb-9985668831fc" containerName="cleanup"
Oct 31 22:00:32.902459 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.902263    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="63aa3078-082d-400e-abfb-9985668831fc" containerName="cleanup"
Oct 31 22:00:32.902459 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.902275    4460 cpu_manager.go:414] "containerMap: removing container" podUID="fb824101-1f12-403b-bf06-1051f6f25d17" containerName="cleanup"
Oct 31 22:00:32.902459 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.902282    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fb824101-1f12-403b-bf06-1051f6f25d17" containerName="cleanup"
Oct 31 22:00:32.902459 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.902313    4460 cpu_manager.go:414] "containerMap: removing container" podUID="0a11ccff-84e3-4179-9f19-f07b86783bd3" containerName="cleanup"
Oct 31 22:00:32.902459 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.902319    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a11ccff-84e3-4179-9f19-f07b86783bd3" containerName="cleanup"
Oct 31 22:00:32.902459 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.902344    4460 cpu_manager.go:414] "containerMap: removing container" podUID="23b20e87-150e-48d7-9da9-d6c81de609d7" containerName="cleanup"
Oct 31 22:00:32.902459 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.902352    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="23b20e87-150e-48d7-9da9-d6c81de609d7" containerName="cleanup"
Oct 31 22:00:32.902459 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.902371    4460 cpu_manager.go:414] "containerMap: removing container" podUID="0864e283-2018-4c62-abeb-809f9dd12ce4" containerName="cleanup"
Oct 31 22:00:32.902459 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.902377    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0864e283-2018-4c62-abeb-809f9dd12ce4" containerName="cleanup"
Oct 31 22:00:32.902459 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.902388    4460 cpu_manager.go:414] "containerMap: removing container" podUID="76255444-0212-46b9-9a6d-571bc0ba8f28" containerName="cleanup"
Oct 31 22:00:32.902459 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.902394    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="76255444-0212-46b9-9a6d-571bc0ba8f28" containerName="cleanup"
Oct 31 22:00:32.902459 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.902409    4460 cpu_manager.go:414] "containerMap: removing container" podUID="229deecb-4066-4ef7-bcf0-70bd5c15df45" containerName="cleanup"
Oct 31 22:00:32.902459 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.902416    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="229deecb-4066-4ef7-bcf0-70bd5c15df45" containerName="cleanup"
Oct 31 22:00:32.902459 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.902436    4460 cpu_manager.go:414] "containerMap: removing container" podUID="093b9d9a-ec69-4552-a889-6f3bccda27cc" containerName="cleanup"
Oct 31 22:00:32.902459 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.902444    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="093b9d9a-ec69-4552-a889-6f3bccda27cc" containerName="cleanup"
Oct 31 22:00:32.903445 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.902469    4460 cpu_manager.go:414] "containerMap: removing container" podUID="12241bee-ccf4-4a34-8e1a-cfec063d543e" containerName="payload-processing"
Oct 31 22:00:32.903445 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.902476    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="12241bee-ccf4-4a34-8e1a-cfec063d543e" containerName="payload-processing"
Oct 31 22:00:32.903445 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.902687    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="093b9d9a-ec69-4552-a889-6f3bccda27cc" containerName="cleanup"
Oct 31 22:00:32.903445 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.902713    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="12241bee-ccf4-4a34-8e1a-cfec063d543e" containerName="payload-processing"
Oct 31 22:00:32.903445 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.902723    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="0864e283-2018-4c62-abeb-809f9dd12ce4" containerName="cleanup"
Oct 31 22:00:32.903445 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.902733    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="229deecb-4066-4ef7-bcf0-70bd5c15df45" containerName="cleanup"
Oct 31 22:00:32.903445 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.902746    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="31d27730-27b7-4bfa-ac38-5a4a3cc13487" containerName="cleanup"
Oct 31 22:00:32.903445 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.902756    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="fb824101-1f12-403b-bf06-1051f6f25d17" containerName="cleanup"
Oct 31 22:00:32.903445 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.902766    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="23b20e87-150e-48d7-9da9-d6c81de609d7" containerName="cleanup"
Oct 31 22:00:32.903445 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.902776    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a11ccff-84e3-4179-9f19-f07b86783bd3" containerName="cleanup"
Oct 31 22:00:32.909988 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.908256    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p"
Oct 31 22:00:32.922990 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.915528    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p"]
Oct 31 22:00:32.938486 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.938453    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-4bd51b\"/\"agc-cred-2a6f00721342\""
Oct 31 22:00:32.938656 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.938627    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-4bd51b\"/\"payload-processing-external-model-e2e--c7d64253-dockercfg-lhtkj\""
Oct 31 22:00:32.938736 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.938659    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-4bd51b\"/\"openshift-service-ca.crt\""
Oct 31 22:00:32.938736 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.938697    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-4bd51b\"/\"kube-root-ca.crt\""
Oct 31 22:00:32.938736 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.938723    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-4bd51b\"/\"routing-overlay\""
Oct 31 22:00:32.938736 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.938734    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-4bd51b\"/\"payload-processing-external-model-plugins-e2e-extmod-a-4bd51b\""
Oct 31 22:00:32.938975 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.938755    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-4bd51b\"/\"e2e-cfl-a73b87-provider-cred\""
Oct 31 22:00:32.989255 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.989218    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/0d3e62a4-6f08-4134-8247-39503caeb51b-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p\" (UID: \"0d3e62a4-6f08-4134-8247-39503caeb51b\") " pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p"
Oct 31 22:00:32.989454 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.989435    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d3e62a4-6f08-4134-8247-39503caeb51b-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p\" (UID: \"0d3e62a4-6f08-4134-8247-39503caeb51b\") " pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p"
Oct 31 22:00:32.989562 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.989543    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kghkp\" (UniqueName: \"kubernetes.io/projected/0d3e62a4-6f08-4134-8247-39503caeb51b-kube-api-access-kghkp\") pod \"payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p\" (UID: \"0d3e62a4-6f08-4134-8247-39503caeb51b\") " pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p"
Oct 31 22:00:32.989623 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:32.989576    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/0d3e62a4-6f08-4134-8247-39503caeb51b-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p\" (UID: \"0d3e62a4-6f08-4134-8247-39503caeb51b\") " pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p"
Oct 31 22:00:33.091123 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:33.091033    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/0d3e62a4-6f08-4134-8247-39503caeb51b-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p\" (UID: \"0d3e62a4-6f08-4134-8247-39503caeb51b\") " pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p"
Oct 31 22:00:33.091295 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:33.091184    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d3e62a4-6f08-4134-8247-39503caeb51b-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p\" (UID: \"0d3e62a4-6f08-4134-8247-39503caeb51b\") " pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p"
Oct 31 22:00:33.091295 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:33.091236    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kghkp\" (UniqueName: \"kubernetes.io/projected/0d3e62a4-6f08-4134-8247-39503caeb51b-kube-api-access-kghkp\") pod \"payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p\" (UID: \"0d3e62a4-6f08-4134-8247-39503caeb51b\") " pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p"
Oct 31 22:00:33.091295 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:33.091253    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/0d3e62a4-6f08-4134-8247-39503caeb51b-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p\" (UID: \"0d3e62a4-6f08-4134-8247-39503caeb51b\") " pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p"
Oct 31 22:00:33.091870 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:33.091850    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/0d3e62a4-6f08-4134-8247-39503caeb51b-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p\" (UID: \"0d3e62a4-6f08-4134-8247-39503caeb51b\") " pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p"
Oct 31 22:00:33.091932 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:33.091914    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d3e62a4-6f08-4134-8247-39503caeb51b-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p\" (UID: \"0d3e62a4-6f08-4134-8247-39503caeb51b\") " pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p"
Oct 31 22:00:33.093667 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:33.093641    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/0d3e62a4-6f08-4134-8247-39503caeb51b-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p\" (UID: \"0d3e62a4-6f08-4134-8247-39503caeb51b\") " pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p"
Oct 31 22:00:33.116462 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:33.116429    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kghkp\" (UniqueName: \"kubernetes.io/projected/0d3e62a4-6f08-4134-8247-39503caeb51b-kube-api-access-kghkp\") pod \"payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p\" (UID: \"0d3e62a4-6f08-4134-8247-39503caeb51b\") " pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p"
Oct 31 22:00:33.229129 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:33.229095    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p"
Oct 31 22:00:33.269782 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:33.269743    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:33.395582 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:33.395501    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:33.542128 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:33.542097    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p"]
Oct 31 22:00:33.542727 ip-10-0-136-157 kubenswrapper[4460]: W1031 22:00:33.542702    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d3e62a4_6f08_4134_8247_39503caeb51b.slice/crio-5af20c528ef1ea54f3464e0990f724bc2745f2745dfa21312f9fb3dcbf4717c7 WatchSource:0}: Error finding container 5af20c528ef1ea54f3464e0990f724bc2745f2745dfa21312f9fb3dcbf4717c7: Status 404 returned error can't find the container with id 5af20c528ef1ea54f3464e0990f724bc2745f2745dfa21312f9fb3dcbf4717c7
Oct 31 22:00:33.572732 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:33.572704    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:33.881778 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:33.881734    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:34.028500 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:34.028465    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:34.074094 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:34.074058    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p" event={"ID":"0d3e62a4-6f08-4134-8247-39503caeb51b","Type":"ContainerStarted","Data":"85a22568a8a134e38acb98674f1a4b8a1a5ac0792b1417dab7a206b16e8dbe20"}
Oct 31 22:00:34.074282 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:34.074116    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p" event={"ID":"0d3e62a4-6f08-4134-8247-39503caeb51b","Type":"ContainerStarted","Data":"5af20c528ef1ea54f3464e0990f724bc2745f2745dfa21312f9fb3dcbf4717c7"}
Oct 31 22:00:34.074329 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:34.074274    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p"
Oct 31 22:00:34.129397 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:34.129343    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p" podStartSLOduration=2.12932607 podStartE2EDuration="2.12932607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:00:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:00:34.126529649 +0000 UTC m=+2674.042839080" watchObservedRunningTime="2026-10-31 22:00:34.12932607 +0000 UTC m=+2674.045635502"
Oct 31 22:00:34.147073 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:34.146978    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:34.263723 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:34.263685    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:34.329399 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:34.329359    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm"]
Oct 31 22:00:34.332996 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:34.332969    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm"
Oct 31 22:00:34.386298 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:34.386265    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:34.644458 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:34.644422    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:34.757305 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:34.757263    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:34.866496 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:34.866462    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:35.694997 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:35.694928    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:35.960129 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:35.960044    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz"]
Oct 31 22:00:35.965197 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:35.965174    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz"
Oct 31 22:00:35.990965 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:35.988753    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz"]
Oct 31 22:00:35.995969 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:35.993266    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-a5affe\"/\"kube-root-ca.crt\""
Oct 31 22:00:35.995969 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:35.993454    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-a5affe\"/\"payload-processing-external-model-e2e--def0cae2-dockercfg-2qggm\""
Oct 31 22:00:35.995969 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:35.993587    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-a5affe\"/\"agc-cred-3855aebbf33b\""
Oct 31 22:00:36.002965 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:36.002180    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-a5affe\"/\"openshift-service-ca.crt\""
Oct 31 22:00:36.002965 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:36.002241    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-a5affe\"/\"routing-overlay\""
Oct 31 22:00:36.002965 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:36.002294    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-a5affe\"/\"payload-processing-external-model-plugins-e2e-extmod-b-a5affe\""
Oct 31 22:00:36.043333 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:36.043302    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/db869980-f710-4c15-b810-391a663fe7fd-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz\" (UID: \"db869980-f710-4c15-b810-391a663fe7fd\") " pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz"
Oct 31 22:00:36.043524 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:36.043380    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/db869980-f710-4c15-b810-391a663fe7fd-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz\" (UID: \"db869980-f710-4c15-b810-391a663fe7fd\") " pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz"
Oct 31 22:00:36.043524 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:36.043483    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqr8d\" (UniqueName: \"kubernetes.io/projected/db869980-f710-4c15-b810-391a663fe7fd-kube-api-access-dqr8d\") pod \"payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz\" (UID: \"db869980-f710-4c15-b810-391a663fe7fd\") " pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz"
Oct 31 22:00:36.043634 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:36.043612    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db869980-f710-4c15-b810-391a663fe7fd-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz\" (UID: \"db869980-f710-4c15-b810-391a663fe7fd\") " pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz"
Oct 31 22:00:36.157023 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:36.156301    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/db869980-f710-4c15-b810-391a663fe7fd-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz\" (UID: \"db869980-f710-4c15-b810-391a663fe7fd\") " pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz"
Oct 31 22:00:36.157023 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:36.156457    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqr8d\" (UniqueName: \"kubernetes.io/projected/db869980-f710-4c15-b810-391a663fe7fd-kube-api-access-dqr8d\") pod \"payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz\" (UID: \"db869980-f710-4c15-b810-391a663fe7fd\") " pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz"
Oct 31 22:00:36.157023 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:36.156605    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db869980-f710-4c15-b810-391a663fe7fd-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz\" (UID: \"db869980-f710-4c15-b810-391a663fe7fd\") " pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz"
Oct 31 22:00:36.157023 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:36.156750    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/db869980-f710-4c15-b810-391a663fe7fd-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz\" (UID: \"db869980-f710-4c15-b810-391a663fe7fd\") " pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz"
Oct 31 22:00:36.187878 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:36.172623    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/db869980-f710-4c15-b810-391a663fe7fd-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz\" (UID: \"db869980-f710-4c15-b810-391a663fe7fd\") " pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz"
Oct 31 22:00:36.187878 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:36.173667    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db869980-f710-4c15-b810-391a663fe7fd-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz\" (UID: \"db869980-f710-4c15-b810-391a663fe7fd\") " pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz"
Oct 31 22:00:36.187878 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:36.175472    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/db869980-f710-4c15-b810-391a663fe7fd-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz\" (UID: \"db869980-f710-4c15-b810-391a663fe7fd\") " pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz"
Oct 31 22:00:36.217545 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:36.217460    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"]
Oct 31 22:00:36.219273 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:36.219235    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94" podUID="a4227bd4-a2ef-4c55-9a6a-a0347e9afa66" containerName="istio-proxy" containerID="cri-o://241ea9f66a5f5bc555331327dee64dc4059704670c4041a76c1e78eaecce2f79" gracePeriod=30
Oct 31 22:00:36.231268 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:36.230782    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqr8d\" (UniqueName: \"kubernetes.io/projected/db869980-f710-4c15-b810-391a663fe7fd-kube-api-access-dqr8d\") pod \"payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz\" (UID: \"db869980-f710-4c15-b810-391a663fe7fd\") " pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz"
Oct 31 22:00:36.306245 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:36.306212    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz"
Oct 31 22:00:36.546993 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:36.546955    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:36.631445 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:36.628763    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz"]
Oct 31 22:00:36.637094 ip-10-0-136-157 kubenswrapper[4460]: W1031 22:00:36.637047    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb869980_f710_4c15_b810_391a663fe7fd.slice/crio-425be77c19c7821c536a74241253cde251b3649a856b141b4713d1a5b6ed53c0 WatchSource:0}: Error finding container 425be77c19c7821c536a74241253cde251b3649a856b141b4713d1a5b6ed53c0: Status 404 returned error can't find the container with id 425be77c19c7821c536a74241253cde251b3649a856b141b4713d1a5b6ed53c0
Oct 31 22:00:37.216184 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:37.216144    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz" event={"ID":"db869980-f710-4c15-b810-391a663fe7fd","Type":"ContainerStarted","Data":"17260aea233abd7984de63e270c9a6d99de8797f6d70c872c41ff7a8f5a92165"}
Oct 31 22:00:37.216670 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:37.216223    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz" event={"ID":"db869980-f710-4c15-b810-391a663fe7fd","Type":"ContainerStarted","Data":"425be77c19c7821c536a74241253cde251b3649a856b141b4713d1a5b6ed53c0"}
Oct 31 22:00:37.216670 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:37.216291    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz"
Oct 31 22:00:37.262393 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:37.262322    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz" podStartSLOduration=2.262309021 podStartE2EDuration="2.262309021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:00:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:00:37.260330505 +0000 UTC m=+2677.176639935" watchObservedRunningTime="2026-10-31 22:00:37.262309021 +0000 UTC m=+2677.178618468"
Oct 31 22:00:38.319091 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:38.319047    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:39.329924 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:39.329886    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm" podUID="81c4b66d-2e79-487c-80cd-69a66d76a3bf" containerName="payload-processing" containerID="cri-o://c28721fec3b2e4bcbaf37f8e3d681964daefbe65d8d286dd44362ea8dbada12d" gracePeriod=25
Oct 31 22:00:39.642769 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:39.642741    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm"
Oct 31 22:00:39.813561 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:39.813526    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81c4b66d-2e79-487c-80cd-69a66d76a3bf-kube-api-access-8qvqb\" (UniqueName: \"kubernetes.io/projected/81c4b66d-2e79-487c-80cd-69a66d76a3bf-kube-api-access-8qvqb\") pod \"81c4b66d-2e79-487c-80cd-69a66d76a3bf\" (UID: \"81c4b66d-2e79-487c-80cd-69a66d76a3bf\") "
Oct 31 22:00:39.813767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:39.813578    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/81c4b66d-2e79-487c-80cd-69a66d76a3bf-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/81c4b66d-2e79-487c-80cd-69a66d76a3bf-routing-overlay-volume\") pod \"81c4b66d-2e79-487c-80cd-69a66d76a3bf\" (UID: \"81c4b66d-2e79-487c-80cd-69a66d76a3bf\") "
Oct 31 22:00:39.813767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:39.813732    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/81c4b66d-2e79-487c-80cd-69a66d76a3bf-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81c4b66d-2e79-487c-80cd-69a66d76a3bf-plugins-config-volume\") pod \"81c4b66d-2e79-487c-80cd-69a66d76a3bf\" (UID: \"81c4b66d-2e79-487c-80cd-69a66d76a3bf\") "
Oct 31 22:00:39.813912 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:39.813804    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81c4b66d-2e79-487c-80cd-69a66d76a3bf-provider-credentials\" (UniqueName: \"kubernetes.io/projected/81c4b66d-2e79-487c-80cd-69a66d76a3bf-provider-credentials\") pod \"81c4b66d-2e79-487c-80cd-69a66d76a3bf\" (UID: \"81c4b66d-2e79-487c-80cd-69a66d76a3bf\") "
Oct 31 22:00:39.814092 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:39.814065    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81c4b66d-2e79-487c-80cd-69a66d76a3bf-routing-overlay-volume" pod "81c4b66d-2e79-487c-80cd-69a66d76a3bf" (UID: "81c4b66d-2e79-487c-80cd-69a66d76a3bf"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:00:39.814161 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:39.814138    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81c4b66d-2e79-487c-80cd-69a66d76a3bf-plugins-config-volume" pod "81c4b66d-2e79-487c-80cd-69a66d76a3bf" (UID: "81c4b66d-2e79-487c-80cd-69a66d76a3bf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:00:39.814336 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:39.814311    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81c4b66d-2e79-487c-80cd-69a66d76a3bf-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:39.814336 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:39.814332    4460 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/81c4b66d-2e79-487c-80cd-69a66d76a3bf-routing-overlay-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:39.815863 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:39.815842    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81c4b66d-2e79-487c-80cd-69a66d76a3bf-kube-api-access-8qvqb" pod "81c4b66d-2e79-487c-80cd-69a66d76a3bf" (UID: "81c4b66d-2e79-487c-80cd-69a66d76a3bf"). InnerVolumeSpecName "kube-api-access-8qvqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:00:39.815966 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:39.815871    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81c4b66d-2e79-487c-80cd-69a66d76a3bf-provider-credentials" pod "81c4b66d-2e79-487c-80cd-69a66d76a3bf" (UID: "81c4b66d-2e79-487c-80cd-69a66d76a3bf"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:00:39.915912 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:39.915803    4460 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/81c4b66d-2e79-487c-80cd-69a66d76a3bf-provider-credentials\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:39.915912 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:39.915843    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qvqb\" (UniqueName: \"kubernetes.io/projected/81c4b66d-2e79-487c-80cd-69a66d76a3bf-kube-api-access-8qvqb\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:40.119028 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:40.118990    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:40.240975 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:40.240929    4460 generic.go:356] "Generic (PLEG): container finished" podID="81c4b66d-2e79-487c-80cd-69a66d76a3bf" containerID="c28721fec3b2e4bcbaf37f8e3d681964daefbe65d8d286dd44362ea8dbada12d" exitCode=0
Oct 31 22:00:40.241153 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:40.240972    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm" event={"ID":"81c4b66d-2e79-487c-80cd-69a66d76a3bf","Type":"ContainerDied","Data":"c28721fec3b2e4bcbaf37f8e3d681964daefbe65d8d286dd44362ea8dbada12d"}
Oct 31 22:00:40.241153 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:40.240991    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm"
Oct 31 22:00:40.241153 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:40.241032    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm" event={"ID":"81c4b66d-2e79-487c-80cd-69a66d76a3bf","Type":"ContainerDied","Data":"dd5227d01a494a36e1cc776d23f35d5398a12214dbdcb175918634b5d48729a2"}
Oct 31 22:00:40.241153 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:40.241056    4460 scope.go:122] "RemoveContainer" containerID="c28721fec3b2e4bcbaf37f8e3d681964daefbe65d8d286dd44362ea8dbada12d"
Oct 31 22:00:40.252507 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:40.252486    4460 scope.go:122] "RemoveContainer" containerID="c28721fec3b2e4bcbaf37f8e3d681964daefbe65d8d286dd44362ea8dbada12d"
Oct 31 22:00:40.252821 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:40.252792    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c28721fec3b2e4bcbaf37f8e3d681964daefbe65d8d286dd44362ea8dbada12d\": container with ID starting with c28721fec3b2e4bcbaf37f8e3d681964daefbe65d8d286dd44362ea8dbada12d not found: ID does not exist" containerID="c28721fec3b2e4bcbaf37f8e3d681964daefbe65d8d286dd44362ea8dbada12d"
Oct 31 22:00:40.252926 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:40.252826    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c28721fec3b2e4bcbaf37f8e3d681964daefbe65d8d286dd44362ea8dbada12d"} err="failed to get container status \"c28721fec3b2e4bcbaf37f8e3d681964daefbe65d8d286dd44362ea8dbada12d\": rpc error: code = NotFound desc = could not find container \"c28721fec3b2e4bcbaf37f8e3d681964daefbe65d8d286dd44362ea8dbada12d\": container with ID starting with c28721fec3b2e4bcbaf37f8e3d681964daefbe65d8d286dd44362ea8dbada12d not found: ID does not exist"
Oct 31 22:00:40.299405 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:40.299355    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm"]
Oct 31 22:00:40.306661 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:40.306627    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-6ff92zlvm"]
Oct 31 22:00:41.396166 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:41.396125    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:41.507101 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:41.507064    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:41.529504 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:41.529476    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 22:00:41.633404 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:41.633369    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:41.641657 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:41.641625    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istio-envoy\") pod \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") "
Oct 31 22:00:41.641820 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:41.641667    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istio-data\") pod \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") "
Oct 31 22:00:41.641820 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:41.641716    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-workload-socket\") pod \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") "
Oct 31 22:00:41.641820 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:41.641791    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-kube-api-access-9p4wr\" (UniqueName: \"kubernetes.io/projected/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-kube-api-access-9p4wr\") pod \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") "
Oct 31 22:00:41.641972 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:41.641879    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-workload-certs\") pod \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") "
Oct 31 22:00:41.641972 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:41.641909    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-credential-socket\") pod \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") "
Oct 31 22:00:41.642577 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:41.642014    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istio-data" pod "a4227bd4-a2ef-4c55-9a6a-a0347e9afa66" (UID: "a4227bd4-a2ef-4c55-9a6a-a0347e9afa66"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:00:41.642577 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:41.642050    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istio-token\" (UniqueName: \"kubernetes.io/projected/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istio-token\") pod \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") "
Oct 31 22:00:41.642577 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:41.642112    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istiod-ca-cert\") pod \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") "
Oct 31 22:00:41.642577 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:41.642111    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-workload-socket" pod "a4227bd4-a2ef-4c55-9a6a-a0347e9afa66" (UID: "a4227bd4-a2ef-4c55-9a6a-a0347e9afa66"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:00:41.642577 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:41.642144    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istio-podinfo\") pod \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\" (UID: \"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66\") "
Oct 31 22:00:41.642577 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:41.642200    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-workload-certs" pod "a4227bd4-a2ef-4c55-9a6a-a0347e9afa66" (UID: "a4227bd4-a2ef-4c55-9a6a-a0347e9afa66"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:00:41.642960 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:41.642579    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istiod-ca-cert" pod "a4227bd4-a2ef-4c55-9a6a-a0347e9afa66" (UID: "a4227bd4-a2ef-4c55-9a6a-a0347e9afa66"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:00:41.642960 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:41.642644    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-credential-socket" pod "a4227bd4-a2ef-4c55-9a6a-a0347e9afa66" (UID: "a4227bd4-a2ef-4c55-9a6a-a0347e9afa66"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:00:41.643161 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:41.643131    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:41.643161 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:41.643162    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:41.643348 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:41.643177    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:41.643348 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:41.643190    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:41.643348 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:41.643203    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:41.644360 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:41.644329    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istio-envoy" pod "a4227bd4-a2ef-4c55-9a6a-a0347e9afa66" (UID: "a4227bd4-a2ef-4c55-9a6a-a0347e9afa66"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:00:41.644479 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:41.644371    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-kube-api-access-9p4wr" pod "a4227bd4-a2ef-4c55-9a6a-a0347e9afa66" (UID: "a4227bd4-a2ef-4c55-9a6a-a0347e9afa66"). InnerVolumeSpecName "kube-api-access-9p4wr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:00:41.644479 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:41.644465    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istio-token" pod "a4227bd4-a2ef-4c55-9a6a-a0347e9afa66" (UID: "a4227bd4-a2ef-4c55-9a6a-a0347e9afa66"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:00:41.644694 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:41.644676    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istio-podinfo" pod "a4227bd4-a2ef-4c55-9a6a-a0347e9afa66" (UID: "a4227bd4-a2ef-4c55-9a6a-a0347e9afa66"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 22:00:41.745070 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:41.745030    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:41.745070 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:41.745063    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9p4wr\" (UniqueName: \"kubernetes.io/projected/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-kube-api-access-9p4wr\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:41.745070 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:41.745075    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:41.745070 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:41.745084    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:00:41.837558 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:41.837519    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:42.226624 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:42.226536    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="81c4b66d-2e79-487c-80cd-69a66d76a3bf" path="/var/lib/kubelet/pods/81c4b66d-2e79-487c-80cd-69a66d76a3bf/volumes"
Oct 31 22:00:42.260593 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:42.260555    4460 generic.go:356] "Generic (PLEG): container finished" podID="a4227bd4-a2ef-4c55-9a6a-a0347e9afa66" containerID="241ea9f66a5f5bc555331327dee64dc4059704670c4041a76c1e78eaecce2f79" exitCode=0
Oct 31 22:00:42.260771 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:42.260597    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"
Oct 31 22:00:42.260771 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:42.260602    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94" event={"ID":"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66","Type":"ContainerDied","Data":"241ea9f66a5f5bc555331327dee64dc4059704670c4041a76c1e78eaecce2f79"}
Oct 31 22:00:42.260771 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:42.260653    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94" event={"ID":"a4227bd4-a2ef-4c55-9a6a-a0347e9afa66","Type":"ContainerDied","Data":"aedc28bc8a2500dfe1903989fbb465f641f1d28f1f5c83227dd7f4cb0512dec0"}
Oct 31 22:00:42.260771 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:42.260683    4460 scope.go:122] "RemoveContainer" containerID="241ea9f66a5f5bc555331327dee64dc4059704670c4041a76c1e78eaecce2f79"
Oct 31 22:00:42.271770 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:42.271744    4460 scope.go:122] "RemoveContainer" containerID="241ea9f66a5f5bc555331327dee64dc4059704670c4041a76c1e78eaecce2f79"
Oct 31 22:00:42.272087 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:42.272066    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"241ea9f66a5f5bc555331327dee64dc4059704670c4041a76c1e78eaecce2f79\": container with ID starting with 241ea9f66a5f5bc555331327dee64dc4059704670c4041a76c1e78eaecce2f79 not found: ID does not exist" containerID="241ea9f66a5f5bc555331327dee64dc4059704670c4041a76c1e78eaecce2f79"
Oct 31 22:00:42.272167 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:42.272094    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"241ea9f66a5f5bc555331327dee64dc4059704670c4041a76c1e78eaecce2f79"} err="failed to get container status \"241ea9f66a5f5bc555331327dee64dc4059704670c4041a76c1e78eaecce2f79\": rpc error: code = NotFound desc = could not find container \"241ea9f66a5f5bc555331327dee64dc4059704670c4041a76c1e78eaecce2f79\": container with ID starting with 241ea9f66a5f5bc555331327dee64dc4059704670c4041a76c1e78eaecce2f79 not found: ID does not exist"
Oct 31 22:00:42.319930 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:42.319898    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"]
Oct 31 22:00:42.323590 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:42.323564    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-0877f5-openshift-default-65f9674c97-65s94"]
Oct 31 22:00:43.531219 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:43.531181    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:44.227120 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:44.227084    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a4227bd4-a2ef-4c55-9a6a-a0347e9afa66" path="/var/lib/kubelet/pods/a4227bd4-a2ef-4c55-9a6a-a0347e9afa66/volumes"
Oct 31 22:00:45.101997 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:45.101964    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p"
Oct 31 22:00:45.261675 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:45.261638    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:46.922695 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:46.922650    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:48.000125 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:48.000089    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:48.131807 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:48.131771    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:48.228377 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:48.228344    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz"
Oct 31 22:00:48.246401 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:48.246375    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:48.458214 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:48.458174    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:48.599182 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:48.599149    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:48.727582 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:48.727498    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:48.907434 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:48.907399    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:49.054379 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:49.054342    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:49.527278 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:49.527234    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:49.659456 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:49.659421    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:50.537625 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:50.537593    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:50.726123 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:50.726088    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:50.841487 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:50.841402    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:50.973753 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:50.973717    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:51.478953 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:00:51.478804    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:00:52.235950 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.235901    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"]
Oct 31 22:00:52.236669 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.236653    4460 cpu_manager.go:414] "containerMap: removing container" podUID="a4227bd4-a2ef-4c55-9a6a-a0347e9afa66" containerName="istio-proxy"
Oct 31 22:00:52.236717 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.236670    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a4227bd4-a2ef-4c55-9a6a-a0347e9afa66" containerName="istio-proxy"
Oct 31 22:00:52.236717 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.236684    4460 cpu_manager.go:414] "containerMap: removing container" podUID="81c4b66d-2e79-487c-80cd-69a66d76a3bf" containerName="payload-processing"
Oct 31 22:00:52.236717 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.236689    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="81c4b66d-2e79-487c-80cd-69a66d76a3bf" containerName="payload-processing"
Oct 31 22:00:52.236868 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.236849    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="a4227bd4-a2ef-4c55-9a6a-a0347e9afa66" containerName="istio-proxy"
Oct 31 22:00:52.236868 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.236861    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="81c4b66d-2e79-487c-80cd-69a66d76a3bf" containerName="payload-processing"
Oct 31 22:00:52.242037 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.242015    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:00:52.263855 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.263826    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"]
Oct 31 22:00:52.271517 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.271493    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-fac043-openshift-default-dockercfg-4n97g\""
Oct 31 22:00:52.304640 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.304612    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-workload-socket\") pod \"e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") " pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:00:52.304782 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.304645    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1df58c25-2f7d-438b-a45d-0ff511a344ce-istio-token\") pod \"e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") " pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:00:52.304782 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.304666    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-credential-socket\") pod \"e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") " pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:00:52.304865 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.304784    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-istio-data\") pod \"e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") " pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:00:52.304865 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.304826    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-istio-envoy\") pod \"e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") " pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:00:52.305019 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.304992    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzjkg\" (UniqueName: \"kubernetes.io/projected/1df58c25-2f7d-438b-a45d-0ff511a344ce-kube-api-access-jzjkg\") pod \"e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") " pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:00:52.305166 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.305147    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1df58c25-2f7d-438b-a45d-0ff511a344ce-istio-podinfo\") pod \"e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") " pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:00:52.305217 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.305185    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1df58c25-2f7d-438b-a45d-0ff511a344ce-istiod-ca-cert\") pod \"e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") " pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:00:52.305304 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.305287    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-workload-certs\") pod \"e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") " pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:00:52.308376 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.308351    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:52.406698 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.406669    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-workload-certs\") pod \"e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") " pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:00:52.406903 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.406713    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-workload-socket\") pod \"e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") " pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:00:52.406903 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.406733    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1df58c25-2f7d-438b-a45d-0ff511a344ce-istio-token\") pod \"e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") " pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:00:52.406903 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.406752    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-credential-socket\") pod \"e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") " pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:00:52.406903 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.406799    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-istio-data\") pod \"e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") " pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:00:52.406903 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.406832    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-istio-envoy\") pod \"e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") " pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:00:52.407277 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.406913    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzjkg\" (UniqueName: \"kubernetes.io/projected/1df58c25-2f7d-438b-a45d-0ff511a344ce-kube-api-access-jzjkg\") pod \"e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") " pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:00:52.407277 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.407105    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1df58c25-2f7d-438b-a45d-0ff511a344ce-istio-podinfo\") pod \"e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") " pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:00:52.407277 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.407141    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1df58c25-2f7d-438b-a45d-0ff511a344ce-istiod-ca-cert\") pod \"e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") " pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:00:52.407277 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.407143    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-workload-certs\") pod \"e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") " pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:00:52.407277 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.407207    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-workload-socket\") pod \"e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") " pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:00:52.407277 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.407262    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-credential-socket\") pod \"e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") " pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:00:52.407568 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.407341    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-istio-data\") pod \"e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") " pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:00:52.407743 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.407726    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1df58c25-2f7d-438b-a45d-0ff511a344ce-istiod-ca-cert\") pod \"e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") " pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:00:52.409238 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.409217    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-istio-envoy\") pod \"e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") " pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:00:52.409542 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.409525    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1df58c25-2f7d-438b-a45d-0ff511a344ce-istio-podinfo\") pod \"e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") " pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:00:52.431194 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.431165    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1df58c25-2f7d-438b-a45d-0ff511a344ce-istio-token\") pod \"e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") " pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:00:52.431456 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.431435    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzjkg\" (UniqueName: \"kubernetes.io/projected/1df58c25-2f7d-438b-a45d-0ff511a344ce-kube-api-access-jzjkg\") pod \"e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") " pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:00:52.558252 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.558224    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:00:52.892087 ip-10-0-136-157 kubenswrapper[4460]: W1031 22:00:52.892055    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1df58c25_2f7d_438b_a45d_0ff511a344ce.slice/crio-b5e7b3721355fedf20e7d97df969e980034ca477c6bebf4d04efcb49c6788ac3 WatchSource:0}: Error finding container b5e7b3721355fedf20e7d97df969e980034ca477c6bebf4d04efcb49c6788ac3: Status 404 returned error can't find the container with id b5e7b3721355fedf20e7d97df969e980034ca477c6bebf4d04efcb49c6788ac3
Oct 31 22:00:52.892550 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.892522    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"]
Oct 31 22:00:52.895263 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.895231    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:00:52.895358 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.895296    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:00:52.895358 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:52.895327    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:00:53.395399 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:53.395357    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh" event={"ID":"1df58c25-2f7d-438b-a45d-0ff511a344ce","Type":"ContainerStarted","Data":"cb370a98dc6f63c7b854ab2021a8f8e9e5eaac2952b5488b1510437734808b91"}
Oct 31 22:00:53.395871 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:53.395418    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh" event={"ID":"1df58c25-2f7d-438b-a45d-0ff511a344ce","Type":"ContainerStarted","Data":"b5e7b3721355fedf20e7d97df969e980034ca477c6bebf4d04efcb49c6788ac3"}
Oct 31 22:00:53.460065 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:53.459975    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh" podStartSLOduration=1.459960502 podStartE2EDuration="1.459960502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:00:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:00:53.456998088 +0000 UTC m=+2693.373307518" watchObservedRunningTime="2026-10-31 22:00:53.459960502 +0000 UTC m=+2693.376269933"
Oct 31 22:00:53.558748 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:53.558717    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:00:53.560045 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:53.560019    4460 patch_prober.go:29] interesting pod/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.200:15021/healthz/ready\": dial tcp 10.133.0.200:15021: connect: connection refused" start-of-body=
Oct 31 22:00:53.560159 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:53.560063    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh" podUID="1df58c25-2f7d-438b-a45d-0ff511a344ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.200:15021/healthz/ready\": dial tcp 10.133.0.200:15021: connect: connection refused"
Oct 31 22:00:54.045029 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:54.044825    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:54.558888 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:54.558855    4460 patch_prober.go:29] interesting pod/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.200:15021/healthz/ready\": dial tcp 10.133.0.200:15021: connect: connection refused" start-of-body=
Oct 31 22:00:54.559416 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:54.558908    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh" podUID="1df58c25-2f7d-438b-a45d-0ff511a344ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.200:15021/healthz/ready\": dial tcp 10.133.0.200:15021: connect: connection refused"
Oct 31 22:00:55.559088 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:55.559051    4460 patch_prober.go:29] interesting pod/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.200:15021/healthz/ready\": dial tcp 10.133.0.200:15021: connect: connection refused" start-of-body=
Oct 31 22:00:55.559501 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:55.559103    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh" podUID="1df58c25-2f7d-438b-a45d-0ff511a344ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.200:15021/healthz/ready\": dial tcp 10.133.0.200:15021: connect: connection refused"
Oct 31 22:00:55.711873 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:55.711836    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:56.559487 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:56.559451    4460 patch_prober.go:29] interesting pod/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.200:15021/healthz/ready\": dial tcp 10.133.0.200:15021: connect: connection refused" start-of-body=
Oct 31 22:00:56.559896 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:56.559508    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh" podUID="1df58c25-2f7d-438b-a45d-0ff511a344ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.200:15021/healthz/ready\": dial tcp 10.133.0.200:15021: connect: connection refused"
Oct 31 22:00:57.446864 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:57.446811    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:57.559303 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:57.559266    4460 patch_prober.go:29] interesting pod/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.200:15021/healthz/ready\": dial tcp 10.133.0.200:15021: connect: connection refused" start-of-body=
Oct 31 22:00:57.559488 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:57.559320    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh" podUID="1df58c25-2f7d-438b-a45d-0ff511a344ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.200:15021/healthz/ready\": dial tcp 10.133.0.200:15021: connect: connection refused"
Oct 31 22:00:58.558855 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:58.558817    4460 patch_prober.go:29] interesting pod/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.200:15021/healthz/ready\": dial tcp 10.133.0.200:15021: connect: connection refused" start-of-body=
Oct 31 22:00:58.559298 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:58.558879    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh" podUID="1df58c25-2f7d-438b-a45d-0ff511a344ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.200:15021/healthz/ready\": dial tcp 10.133.0.200:15021: connect: connection refused"
Oct 31 22:00:59.115005 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:59.114966    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:00:59.399686 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:59.399591    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 22:00:59.559711 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:59.559666    4460 patch_prober.go:29] interesting pod/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.200:15021/healthz/ready\": dial tcp 10.133.0.200:15021: connect: connection refused" start-of-body=
Oct 31 22:00:59.560154 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:00:59.559743    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh" podUID="1df58c25-2f7d-438b-a45d-0ff511a344ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.200:15021/healthz/ready\": dial tcp 10.133.0.200:15021: connect: connection refused"
Oct 31 22:01:00.564018 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:00.563979    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:01:00.565025 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:00.564993    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:01:00.566345 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:00.566323    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:01:00.761980 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:00.761949    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ssl9_26069598-5dc4-4c96-b55f-dd2f09884419/ovn-acl-logging/0.log"
Oct 31 22:01:00.769042 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:00.769013    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ssl9_26069598-5dc4-4c96-b55f-dd2f09884419/ovn-acl-logging/0.log"
Oct 31 22:01:00.972367 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:00.972275    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:02.617891 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:02.617855    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:04.522401 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:04.522362    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:06.316822 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:06.316784    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:08.015829 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:08.015789    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:09.853445 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:09.853407    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:11.619361 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:11.619328    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:13.337451 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:13.337418    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:15.000289 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:15.000252    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:16.630424 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:16.630385    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:18.413966 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:18.413905    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:20.213838 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:20.213799    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:20.956204 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:20.956166    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:21.100968 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:21.100839    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:21.498965 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:01:21.494241    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:01:22.111584 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:22.111553    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:23.827388 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:23.827353    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:25.575184 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:25.575144    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:27.329730 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:27.329697    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:29.046421 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:29.046388    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:30.709692 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:30.709649    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:32.325448 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:32.325416    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:32.771189 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:32.771156    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p"]
Oct 31 22:01:32.850689 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:32.850628    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:32.959893 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:32.959854    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:33.339001 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:33.338966    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:33.603639 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:33.603555    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:34.095626 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:34.095590    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:34.224980 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:34.224921    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:34.510380 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:34.510347    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc"]
Oct 31 22:01:34.515741 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:34.515710    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:34.519113 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:34.519090    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc"]
Oct 31 22:01:34.519232 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:34.519170    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc"
Oct 31 22:01:34.535827 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:34.535805    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-4bd51b\"/\"agc-cred-e47720795e77\""
Oct 31 22:01:34.535967 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:34.535877    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-4bd51b\"/\"payload-processing-external-model-e2e--c7d64253-dockercfg-k86wf\""
Oct 31 22:01:34.668992 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:34.668956    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:34.669594 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:34.669555    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7791402e-4309-4cd8-a527-ec6263dc78a4-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc\" (UID: \"7791402e-4309-4cd8-a527-ec6263dc78a4\") " pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc"
Oct 31 22:01:34.669800 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:34.669781    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/7791402e-4309-4cd8-a527-ec6263dc78a4-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc\" (UID: \"7791402e-4309-4cd8-a527-ec6263dc78a4\") " pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc"
Oct 31 22:01:34.669874 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:34.669822    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jc7hr\" (UniqueName: \"kubernetes.io/projected/7791402e-4309-4cd8-a527-ec6263dc78a4-kube-api-access-jc7hr\") pod \"payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc\" (UID: \"7791402e-4309-4cd8-a527-ec6263dc78a4\") " pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc"
Oct 31 22:01:34.670167 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:34.670140    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7791402e-4309-4cd8-a527-ec6263dc78a4-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc\" (UID: \"7791402e-4309-4cd8-a527-ec6263dc78a4\") " pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc"
Oct 31 22:01:34.775018 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:34.774892    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/7791402e-4309-4cd8-a527-ec6263dc78a4-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc\" (UID: \"7791402e-4309-4cd8-a527-ec6263dc78a4\") " pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc"
Oct 31 22:01:34.775715 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:34.775681    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jc7hr\" (UniqueName: \"kubernetes.io/projected/7791402e-4309-4cd8-a527-ec6263dc78a4-kube-api-access-jc7hr\") pod \"payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc\" (UID: \"7791402e-4309-4cd8-a527-ec6263dc78a4\") " pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc"
Oct 31 22:01:34.776163 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:34.776141    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7791402e-4309-4cd8-a527-ec6263dc78a4-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc\" (UID: \"7791402e-4309-4cd8-a527-ec6263dc78a4\") " pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc"
Oct 31 22:01:34.777671 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:34.777647    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7791402e-4309-4cd8-a527-ec6263dc78a4-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc\" (UID: \"7791402e-4309-4cd8-a527-ec6263dc78a4\") " pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc"
Oct 31 22:01:34.781970 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:01:34.775436    4460 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-4bd51b/agc-cred-e47720795e77: secret "agc-cred-e47720795e77" not found
Oct 31 22:01:34.781970 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:01:34.778140    4460 projected.go:196] Error preparing data for projected volume provider-credentials for pod ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc: secret "agc-cred-e47720795e77" not found
Oct 31 22:01:34.781970 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:01:34.778256    4460 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7791402e-4309-4cd8-a527-ec6263dc78a4-provider-credentials podName:7791402e-4309-4cd8-a527-ec6263dc78a4 nodeName:}" failed. No retries permitted until 2026-10-31 22:01:35.278221586 +0000 UTC m=+2735.194531011 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/7791402e-4309-4cd8-a527-ec6263dc78a4-provider-credentials") pod "payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc" (UID: "7791402e-4309-4cd8-a527-ec6263dc78a4") : secret "agc-cred-e47720795e77" not found
Oct 31 22:01:34.781970 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:34.779325    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7791402e-4309-4cd8-a527-ec6263dc78a4-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc\" (UID: \"7791402e-4309-4cd8-a527-ec6263dc78a4\") " pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc"
Oct 31 22:01:34.782288 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:34.777548    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7791402e-4309-4cd8-a527-ec6263dc78a4-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc\" (UID: \"7791402e-4309-4cd8-a527-ec6263dc78a4\") " pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc"
Oct 31 22:01:34.805957 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:34.805875    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc7hr\" (UniqueName: \"kubernetes.io/projected/7791402e-4309-4cd8-a527-ec6263dc78a4-kube-api-access-jc7hr\") pod \"payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc\" (UID: \"7791402e-4309-4cd8-a527-ec6263dc78a4\") " pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc"
Oct 31 22:01:34.882978 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:34.882933    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc"]
Oct 31 22:01:34.883360 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:01:34.883331    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[provider-credentials], unattached volumes=[], failed to process volumes=[]: context canceled" pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc" podUID="7791402e-4309-4cd8-a527-ec6263dc78a4"
Oct 31 22:01:34.961612 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:34.961574    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:35.094950 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:35.094845    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:35.284967 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:35.284709    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/7791402e-4309-4cd8-a527-ec6263dc78a4-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc\" (UID: \"7791402e-4309-4cd8-a527-ec6263dc78a4\") " pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc"
Oct 31 22:01:35.309962 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:01:35.285872    4460 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-4bd51b/agc-cred-e47720795e77: secret "agc-cred-e47720795e77" not found
Oct 31 22:01:35.309962 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:01:35.285900    4460 projected.go:196] Error preparing data for projected volume provider-credentials for pod ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc: secret "agc-cred-e47720795e77" not found
Oct 31 22:01:35.309962 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:01:35.285988    4460 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7791402e-4309-4cd8-a527-ec6263dc78a4-provider-credentials podName:7791402e-4309-4cd8-a527-ec6263dc78a4 nodeName:}" failed. No retries permitted until 2026-10-31 22:01:36.285968603 +0000 UTC m=+2736.202278025 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/7791402e-4309-4cd8-a527-ec6263dc78a4-provider-credentials") pod "payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc" (UID: "7791402e-4309-4cd8-a527-ec6263dc78a4") : secret "agc-cred-e47720795e77" not found
Oct 31 22:01:35.535311 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:35.535268    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:35.823161 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:35.823078    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc"
Oct 31 22:01:35.831079 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:35.831051    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:35.832143 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:35.832123    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc"
Oct 31 22:01:35.895639 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:35.895605    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7791402e-4309-4cd8-a527-ec6263dc78a4-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7791402e-4309-4cd8-a527-ec6263dc78a4-routing-overlay-volume\") pod \"7791402e-4309-4cd8-a527-ec6263dc78a4\" (UID: \"7791402e-4309-4cd8-a527-ec6263dc78a4\") "
Oct 31 22:01:35.895816 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:35.895644    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7791402e-4309-4cd8-a527-ec6263dc78a4-kube-api-access-jc7hr\" (UniqueName: \"kubernetes.io/projected/7791402e-4309-4cd8-a527-ec6263dc78a4-kube-api-access-jc7hr\") pod \"7791402e-4309-4cd8-a527-ec6263dc78a4\" (UID: \"7791402e-4309-4cd8-a527-ec6263dc78a4\") "
Oct 31 22:01:35.895870 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:35.895806    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7791402e-4309-4cd8-a527-ec6263dc78a4-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7791402e-4309-4cd8-a527-ec6263dc78a4-plugins-config-volume\") pod \"7791402e-4309-4cd8-a527-ec6263dc78a4\" (UID: \"7791402e-4309-4cd8-a527-ec6263dc78a4\") "
Oct 31 22:01:35.896074 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:35.896048    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7791402e-4309-4cd8-a527-ec6263dc78a4-routing-overlay-volume" pod "7791402e-4309-4cd8-a527-ec6263dc78a4" (UID: "7791402e-4309-4cd8-a527-ec6263dc78a4"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:01:35.896237 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:35.896212    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7791402e-4309-4cd8-a527-ec6263dc78a4-plugins-config-volume" pod "7791402e-4309-4cd8-a527-ec6263dc78a4" (UID: "7791402e-4309-4cd8-a527-ec6263dc78a4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:01:35.896529 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:35.896510    4460 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7791402e-4309-4cd8-a527-ec6263dc78a4-routing-overlay-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:01:35.896600 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:35.896534    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7791402e-4309-4cd8-a527-ec6263dc78a4-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:01:35.898074 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:35.898053    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7791402e-4309-4cd8-a527-ec6263dc78a4-kube-api-access-jc7hr" pod "7791402e-4309-4cd8-a527-ec6263dc78a4" (UID: "7791402e-4309-4cd8-a527-ec6263dc78a4"). InnerVolumeSpecName "kube-api-access-jc7hr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:01:35.952043 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:35.945787    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs"]
Oct 31 22:01:36.006266 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:36.006226    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jc7hr\" (UniqueName: \"kubernetes.io/projected/7791402e-4309-4cd8-a527-ec6263dc78a4-kube-api-access-jc7hr\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:01:36.007895 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:36.007862    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:36.117736 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:36.117654    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:36.235730 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:36.235702    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:36.310278 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:36.310236    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/7791402e-4309-4cd8-a527-ec6263dc78a4-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc\" (UID: \"7791402e-4309-4cd8-a527-ec6263dc78a4\") " pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc"
Oct 31 22:01:36.310609 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:01:36.310582    4460 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-4bd51b/agc-cred-e47720795e77: secret "agc-cred-e47720795e77" not found
Oct 31 22:01:36.310609 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:01:36.310609    4460 projected.go:196] Error preparing data for projected volume provider-credentials for pod ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc: secret "agc-cred-e47720795e77" not found
Oct 31 22:01:36.310766 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:01:36.310675    4460 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7791402e-4309-4cd8-a527-ec6263dc78a4-provider-credentials podName:7791402e-4309-4cd8-a527-ec6263dc78a4 nodeName:}" failed. No retries permitted until 2026-10-31 22:01:38.310653846 +0000 UTC m=+2738.226963259 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/7791402e-4309-4cd8-a527-ec6263dc78a4-provider-credentials") pod "payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc" (UID: "7791402e-4309-4cd8-a527-ec6263dc78a4") : secret "agc-cred-e47720795e77" not found
Oct 31 22:01:36.381908 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:36.381810    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:36.565188 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:36.565144    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:36.701501 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:36.701424    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:36.811488 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:36.811450    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:36.830450 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:36.830419    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc"
Oct 31 22:01:36.918904 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:36.918865    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc"]
Oct 31 22:01:36.922858 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:36.922826    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-5689j9sxc"]
Oct 31 22:01:36.981296 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:36.981218    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:37.042234 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:37.042141    4460 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/7791402e-4309-4cd8-a527-ec6263dc78a4-provider-credentials\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:01:37.623580 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:37.623531    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:37.738337 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:37.738297    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:37.772399 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:37.772342    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p" podUID="0d3e62a4-6f08-4134-8247-39503caeb51b" containerName="payload-processing" containerID="cri-o://85a22568a8a134e38acb98674f1a4b8a1a5ac0792b1417dab7a206b16e8dbe20" gracePeriod=25
Oct 31 22:01:37.931091 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:37.931053    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:38.191660 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:38.191633    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p"
Oct 31 22:01:38.227332 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:38.227298    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7791402e-4309-4cd8-a527-ec6263dc78a4" path="/var/lib/kubelet/pods/7791402e-4309-4cd8-a527-ec6263dc78a4/volumes"
Oct 31 22:01:38.270918 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:38.270886    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0d3e62a4-6f08-4134-8247-39503caeb51b-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/0d3e62a4-6f08-4134-8247-39503caeb51b-routing-overlay-volume\") pod \"0d3e62a4-6f08-4134-8247-39503caeb51b\" (UID: \"0d3e62a4-6f08-4134-8247-39503caeb51b\") "
Oct 31 22:01:38.271128 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:38.270957    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d3e62a4-6f08-4134-8247-39503caeb51b-kube-api-access-kghkp\" (UniqueName: \"kubernetes.io/projected/0d3e62a4-6f08-4134-8247-39503caeb51b-kube-api-access-kghkp\") pod \"0d3e62a4-6f08-4134-8247-39503caeb51b\" (UID: \"0d3e62a4-6f08-4134-8247-39503caeb51b\") "
Oct 31 22:01:38.271219 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:38.271127    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d3e62a4-6f08-4134-8247-39503caeb51b-provider-credentials\" (UniqueName: \"kubernetes.io/projected/0d3e62a4-6f08-4134-8247-39503caeb51b-provider-credentials\") pod \"0d3e62a4-6f08-4134-8247-39503caeb51b\" (UID: \"0d3e62a4-6f08-4134-8247-39503caeb51b\") "
Oct 31 22:01:38.271350 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:38.271324    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0d3e62a4-6f08-4134-8247-39503caeb51b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d3e62a4-6f08-4134-8247-39503caeb51b-plugins-config-volume\") pod \"0d3e62a4-6f08-4134-8247-39503caeb51b\" (UID: \"0d3e62a4-6f08-4134-8247-39503caeb51b\") "
Oct 31 22:01:38.271460 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:38.271342    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d3e62a4-6f08-4134-8247-39503caeb51b-routing-overlay-volume" pod "0d3e62a4-6f08-4134-8247-39503caeb51b" (UID: "0d3e62a4-6f08-4134-8247-39503caeb51b"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:01:38.272619 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:38.271826    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d3e62a4-6f08-4134-8247-39503caeb51b-plugins-config-volume" pod "0d3e62a4-6f08-4134-8247-39503caeb51b" (UID: "0d3e62a4-6f08-4134-8247-39503caeb51b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:01:38.272619 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:38.272539    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d3e62a4-6f08-4134-8247-39503caeb51b-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:01:38.272619 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:38.272562    4460 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/0d3e62a4-6f08-4134-8247-39503caeb51b-routing-overlay-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:01:38.273889 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:38.273829    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d3e62a4-6f08-4134-8247-39503caeb51b-provider-credentials" pod "0d3e62a4-6f08-4134-8247-39503caeb51b" (UID: "0d3e62a4-6f08-4134-8247-39503caeb51b"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:01:38.274024 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:38.273882    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d3e62a4-6f08-4134-8247-39503caeb51b-kube-api-access-kghkp" pod "0d3e62a4-6f08-4134-8247-39503caeb51b" (UID: "0d3e62a4-6f08-4134-8247-39503caeb51b"). InnerVolumeSpecName "kube-api-access-kghkp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:01:38.376472 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:38.376431    4460 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/0d3e62a4-6f08-4134-8247-39503caeb51b-provider-credentials\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:01:38.376472 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:38.376471    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kghkp\" (UniqueName: \"kubernetes.io/projected/0d3e62a4-6f08-4134-8247-39503caeb51b-kube-api-access-kghkp\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:01:38.887465 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:38.887431    4460 generic.go:356] "Generic (PLEG): container finished" podID="0d3e62a4-6f08-4134-8247-39503caeb51b" containerID="85a22568a8a134e38acb98674f1a4b8a1a5ac0792b1417dab7a206b16e8dbe20" exitCode=0
Oct 31 22:01:38.887873 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:38.887494    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p"
Oct 31 22:01:38.887873 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:38.887525    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p" event={"ID":"0d3e62a4-6f08-4134-8247-39503caeb51b","Type":"ContainerDied","Data":"85a22568a8a134e38acb98674f1a4b8a1a5ac0792b1417dab7a206b16e8dbe20"}
Oct 31 22:01:38.887873 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:38.887622    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p" event={"ID":"0d3e62a4-6f08-4134-8247-39503caeb51b","Type":"ContainerDied","Data":"5af20c528ef1ea54f3464e0990f724bc2745f2745dfa21312f9fb3dcbf4717c7"}
Oct 31 22:01:38.887873 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:38.887653    4460 scope.go:122] "RemoveContainer" containerID="85a22568a8a134e38acb98674f1a4b8a1a5ac0792b1417dab7a206b16e8dbe20"
Oct 31 22:01:38.894358 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:38.894327    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:38.899443 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:38.899426    4460 scope.go:122] "RemoveContainer" containerID="85a22568a8a134e38acb98674f1a4b8a1a5ac0792b1417dab7a206b16e8dbe20"
Oct 31 22:01:38.899714 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:01:38.899693    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85a22568a8a134e38acb98674f1a4b8a1a5ac0792b1417dab7a206b16e8dbe20\": container with ID starting with 85a22568a8a134e38acb98674f1a4b8a1a5ac0792b1417dab7a206b16e8dbe20 not found: ID does not exist" containerID="85a22568a8a134e38acb98674f1a4b8a1a5ac0792b1417dab7a206b16e8dbe20"
Oct 31 22:01:38.899779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:38.899727    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85a22568a8a134e38acb98674f1a4b8a1a5ac0792b1417dab7a206b16e8dbe20"} err="failed to get container status \"85a22568a8a134e38acb98674f1a4b8a1a5ac0792b1417dab7a206b16e8dbe20\": rpc error: code = NotFound desc = could not find container \"85a22568a8a134e38acb98674f1a4b8a1a5ac0792b1417dab7a206b16e8dbe20\": container with ID starting with 85a22568a8a134e38acb98674f1a4b8a1a5ac0792b1417dab7a206b16e8dbe20 not found: ID does not exist"
Oct 31 22:01:38.946707 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:38.946683    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p"]
Oct 31 22:01:38.959952 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:38.959916    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-a-4bd51b/payload-processing-external-model-e2e-extmod-a-4bd51b-86c7gc49p"]
Oct 31 22:01:39.631533 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:39.631496    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:40.228834 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:40.228798    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0d3e62a4-6f08-4134-8247-39503caeb51b" path="/var/lib/kubelet/pods/0d3e62a4-6f08-4134-8247-39503caeb51b/volumes"
Oct 31 22:01:40.946540 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:40.946498    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs" podUID="231459dd-b722-4b40-ac48-a0d2271a3dcf" containerName="payload-processing" containerID="cri-o://c45048664b7a21ff780b2f99ae21eda33407f7347367dc51389ec8eb69cbeec7" gracePeriod=25
Oct 31 22:01:41.169626 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:41.169587    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:41.259845 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:41.259815    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs"
Oct 31 22:01:41.289105 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:41.289068    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:41.357948 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:41.357905    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/231459dd-b722-4b40-ac48-a0d2271a3dcf-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/231459dd-b722-4b40-ac48-a0d2271a3dcf-plugins-config-volume\") pod \"231459dd-b722-4b40-ac48-a0d2271a3dcf\" (UID: \"231459dd-b722-4b40-ac48-a0d2271a3dcf\") "
Oct 31 22:01:41.358148 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:41.358092    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/231459dd-b722-4b40-ac48-a0d2271a3dcf-provider-credentials\" (UniqueName: \"kubernetes.io/projected/231459dd-b722-4b40-ac48-a0d2271a3dcf-provider-credentials\") pod \"231459dd-b722-4b40-ac48-a0d2271a3dcf\" (UID: \"231459dd-b722-4b40-ac48-a0d2271a3dcf\") "
Oct 31 22:01:41.358148 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:41.358140    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/231459dd-b722-4b40-ac48-a0d2271a3dcf-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/231459dd-b722-4b40-ac48-a0d2271a3dcf-routing-overlay-volume\") pod \"231459dd-b722-4b40-ac48-a0d2271a3dcf\" (UID: \"231459dd-b722-4b40-ac48-a0d2271a3dcf\") "
Oct 31 22:01:41.358293 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:41.358257    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/231459dd-b722-4b40-ac48-a0d2271a3dcf-kube-api-access-z8rzd\" (UniqueName: \"kubernetes.io/projected/231459dd-b722-4b40-ac48-a0d2271a3dcf-kube-api-access-z8rzd\") pod \"231459dd-b722-4b40-ac48-a0d2271a3dcf\" (UID: \"231459dd-b722-4b40-ac48-a0d2271a3dcf\") "
Oct 31 22:01:41.358538 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:41.358399    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/231459dd-b722-4b40-ac48-a0d2271a3dcf-plugins-config-volume" pod "231459dd-b722-4b40-ac48-a0d2271a3dcf" (UID: "231459dd-b722-4b40-ac48-a0d2271a3dcf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:01:41.358635 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:41.358596    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/231459dd-b722-4b40-ac48-a0d2271a3dcf-routing-overlay-volume" pod "231459dd-b722-4b40-ac48-a0d2271a3dcf" (UID: "231459dd-b722-4b40-ac48-a0d2271a3dcf"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:01:41.358913 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:41.358895    4460 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/231459dd-b722-4b40-ac48-a0d2271a3dcf-routing-overlay-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:01:41.358981 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:41.358918    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/231459dd-b722-4b40-ac48-a0d2271a3dcf-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:01:41.360460 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:41.360424    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/231459dd-b722-4b40-ac48-a0d2271a3dcf-kube-api-access-z8rzd" pod "231459dd-b722-4b40-ac48-a0d2271a3dcf" (UID: "231459dd-b722-4b40-ac48-a0d2271a3dcf"). InnerVolumeSpecName "kube-api-access-z8rzd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:01:41.360795 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:41.360774    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/231459dd-b722-4b40-ac48-a0d2271a3dcf-provider-credentials" pod "231459dd-b722-4b40-ac48-a0d2271a3dcf" (UID: "231459dd-b722-4b40-ac48-a0d2271a3dcf"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:01:41.402247 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:41.402208    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:41.460660 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:41.460548    4460 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/231459dd-b722-4b40-ac48-a0d2271a3dcf-provider-credentials\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:01:41.460660 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:41.460575    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8rzd\" (UniqueName: \"kubernetes.io/projected/231459dd-b722-4b40-ac48-a0d2271a3dcf-kube-api-access-z8rzd\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:01:41.514255 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:41.514204    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:41.911489 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:41.911457    4460 generic.go:356] "Generic (PLEG): container finished" podID="231459dd-b722-4b40-ac48-a0d2271a3dcf" containerID="c45048664b7a21ff780b2f99ae21eda33407f7347367dc51389ec8eb69cbeec7" exitCode=0
Oct 31 22:01:41.911673 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:41.911523    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs"
Oct 31 22:01:41.911673 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:41.911545    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs" event={"ID":"231459dd-b722-4b40-ac48-a0d2271a3dcf","Type":"ContainerDied","Data":"c45048664b7a21ff780b2f99ae21eda33407f7347367dc51389ec8eb69cbeec7"}
Oct 31 22:01:41.911673 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:41.911623    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs" event={"ID":"231459dd-b722-4b40-ac48-a0d2271a3dcf","Type":"ContainerDied","Data":"dd43368ec17f3ab07a041093cc055bb537b87f38ae0b49c34cc3e7268e17728a"}
Oct 31 22:01:41.911673 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:41.911646    4460 scope.go:122] "RemoveContainer" containerID="c45048664b7a21ff780b2f99ae21eda33407f7347367dc51389ec8eb69cbeec7"
Oct 31 22:01:41.923288 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:41.923270    4460 scope.go:122] "RemoveContainer" containerID="c45048664b7a21ff780b2f99ae21eda33407f7347367dc51389ec8eb69cbeec7"
Oct 31 22:01:41.923557 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:01:41.923538    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c45048664b7a21ff780b2f99ae21eda33407f7347367dc51389ec8eb69cbeec7\": container with ID starting with c45048664b7a21ff780b2f99ae21eda33407f7347367dc51389ec8eb69cbeec7 not found: ID does not exist" containerID="c45048664b7a21ff780b2f99ae21eda33407f7347367dc51389ec8eb69cbeec7"
Oct 31 22:01:41.923601 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:41.923568    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c45048664b7a21ff780b2f99ae21eda33407f7347367dc51389ec8eb69cbeec7"} err="failed to get container status \"c45048664b7a21ff780b2f99ae21eda33407f7347367dc51389ec8eb69cbeec7\": rpc error: code = NotFound desc = could not find container \"c45048664b7a21ff780b2f99ae21eda33407f7347367dc51389ec8eb69cbeec7\": container with ID starting with c45048664b7a21ff780b2f99ae21eda33407f7347367dc51389ec8eb69cbeec7 not found: ID does not exist"
Oct 31 22:01:41.970006 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:41.969980    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs"]
Oct 31 22:01:41.983276 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:41.983251    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-7c9fh5mxs"]
Oct 31 22:01:42.227640 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:42.227561    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="231459dd-b722-4b40-ac48-a0d2271a3dcf" path="/var/lib/kubelet/pods/231459dd-b722-4b40-ac48-a0d2271a3dcf/volumes"
Oct 31 22:01:43.094064 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:43.093935    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:44.817693 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:44.817659    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:46.514593 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:46.514549    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:46.752373 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:46.752333    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:46.870594 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:46.870496    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:47.759527 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:47.759494    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:47.876267 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:47.876230    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:48.460468 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:48.460432    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:48.681298 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:48.681250    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:48.794321 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:48.794277    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:49.238291 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:49.238191    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:49.964225 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:49.964168    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:50.166477 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:50.166442    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-75fdc85848-47q46"]
Oct 31 22:01:50.167244 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:50.167227    4460 cpu_manager.go:414] "containerMap: removing container" podUID="231459dd-b722-4b40-ac48-a0d2271a3dcf" containerName="payload-processing"
Oct 31 22:01:50.167300 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:50.167246    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="231459dd-b722-4b40-ac48-a0d2271a3dcf" containerName="payload-processing"
Oct 31 22:01:50.167300 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:50.167272    4460 cpu_manager.go:414] "containerMap: removing container" podUID="0d3e62a4-6f08-4134-8247-39503caeb51b" containerName="payload-processing"
Oct 31 22:01:50.167300 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:50.167277    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d3e62a4-6f08-4134-8247-39503caeb51b" containerName="payload-processing"
Oct 31 22:01:50.167399 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:50.167376    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="231459dd-b722-4b40-ac48-a0d2271a3dcf" containerName="payload-processing"
Oct 31 22:01:50.167399 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:50.167391    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d3e62a4-6f08-4134-8247-39503caeb51b" containerName="payload-processing"
Oct 31 22:01:50.172679 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:50.172656    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-75fdc85848-47q46"
Oct 31 22:01:50.191494 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:50.191459    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-a041b65867da\""
Oct 31 22:01:50.192583 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:50.192555    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-75fdc85848-47q46"]
Oct 31 22:01:50.201475 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:50.201436    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-tgxz6\""
Oct 31 22:01:50.201985 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:50.201965    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"openshift-service-ca.crt\""
Oct 31 22:01:50.202111 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:50.202036    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-plugins\""
Oct 31 22:01:50.202184 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:50.202150    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"kube-root-ca.crt\""
Oct 31 22:01:50.205150 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:50.205133    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"routing-overlay\""
Oct 31 22:01:50.213169 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:50.213139    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/cccd0fb2-40d7-4dc3-8170-54e310bf1ce5-provider-credentials\") pod \"payload-processing-external-model-75fdc85848-47q46\" (UID: \"cccd0fb2-40d7-4dc3-8170-54e310bf1ce5\") " pod="models-as-a-service/payload-processing-external-model-75fdc85848-47q46"
Oct 31 22:01:50.213261 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:50.213190    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cccd0fb2-40d7-4dc3-8170-54e310bf1ce5-plugins-config-volume\") pod \"payload-processing-external-model-75fdc85848-47q46\" (UID: \"cccd0fb2-40d7-4dc3-8170-54e310bf1ce5\") " pod="models-as-a-service/payload-processing-external-model-75fdc85848-47q46"
Oct 31 22:01:50.213446 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:50.213417    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/cccd0fb2-40d7-4dc3-8170-54e310bf1ce5-routing-overlay-volume\") pod \"payload-processing-external-model-75fdc85848-47q46\" (UID: \"cccd0fb2-40d7-4dc3-8170-54e310bf1ce5\") " pod="models-as-a-service/payload-processing-external-model-75fdc85848-47q46"
Oct 31 22:01:50.213547 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:50.213534    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc48d\" (UniqueName: \"kubernetes.io/projected/cccd0fb2-40d7-4dc3-8170-54e310bf1ce5-kube-api-access-wc48d\") pod \"payload-processing-external-model-75fdc85848-47q46\" (UID: \"cccd0fb2-40d7-4dc3-8170-54e310bf1ce5\") " pod="models-as-a-service/payload-processing-external-model-75fdc85848-47q46"
Oct 31 22:01:50.244893 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:50.244808    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:50.315499 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:50.315462    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/cccd0fb2-40d7-4dc3-8170-54e310bf1ce5-routing-overlay-volume\") pod \"payload-processing-external-model-75fdc85848-47q46\" (UID: \"cccd0fb2-40d7-4dc3-8170-54e310bf1ce5\") " pod="models-as-a-service/payload-processing-external-model-75fdc85848-47q46"
Oct 31 22:01:50.315684 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:50.315545    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc48d\" (UniqueName: \"kubernetes.io/projected/cccd0fb2-40d7-4dc3-8170-54e310bf1ce5-kube-api-access-wc48d\") pod \"payload-processing-external-model-75fdc85848-47q46\" (UID: \"cccd0fb2-40d7-4dc3-8170-54e310bf1ce5\") " pod="models-as-a-service/payload-processing-external-model-75fdc85848-47q46"
Oct 31 22:01:50.315684 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:50.315663    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/cccd0fb2-40d7-4dc3-8170-54e310bf1ce5-provider-credentials\") pod \"payload-processing-external-model-75fdc85848-47q46\" (UID: \"cccd0fb2-40d7-4dc3-8170-54e310bf1ce5\") " pod="models-as-a-service/payload-processing-external-model-75fdc85848-47q46"
Oct 31 22:01:50.315822 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:50.315688    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cccd0fb2-40d7-4dc3-8170-54e310bf1ce5-plugins-config-volume\") pod \"payload-processing-external-model-75fdc85848-47q46\" (UID: \"cccd0fb2-40d7-4dc3-8170-54e310bf1ce5\") " pod="models-as-a-service/payload-processing-external-model-75fdc85848-47q46"
Oct 31 22:01:50.316426 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:50.316392    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/cccd0fb2-40d7-4dc3-8170-54e310bf1ce5-routing-overlay-volume\") pod \"payload-processing-external-model-75fdc85848-47q46\" (UID: \"cccd0fb2-40d7-4dc3-8170-54e310bf1ce5\") " pod="models-as-a-service/payload-processing-external-model-75fdc85848-47q46"
Oct 31 22:01:50.316547 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:50.316404    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cccd0fb2-40d7-4dc3-8170-54e310bf1ce5-plugins-config-volume\") pod \"payload-processing-external-model-75fdc85848-47q46\" (UID: \"cccd0fb2-40d7-4dc3-8170-54e310bf1ce5\") " pod="models-as-a-service/payload-processing-external-model-75fdc85848-47q46"
Oct 31 22:01:50.318227 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:50.318196    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/cccd0fb2-40d7-4dc3-8170-54e310bf1ce5-provider-credentials\") pod \"payload-processing-external-model-75fdc85848-47q46\" (UID: \"cccd0fb2-40d7-4dc3-8170-54e310bf1ce5\") " pod="models-as-a-service/payload-processing-external-model-75fdc85848-47q46"
Oct 31 22:01:50.341177 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:50.341145    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc48d\" (UniqueName: \"kubernetes.io/projected/cccd0fb2-40d7-4dc3-8170-54e310bf1ce5-kube-api-access-wc48d\") pod \"payload-processing-external-model-75fdc85848-47q46\" (UID: \"cccd0fb2-40d7-4dc3-8170-54e310bf1ce5\") " pod="models-as-a-service/payload-processing-external-model-75fdc85848-47q46"
Oct 31 22:01:50.490263 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:50.490224    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-75fdc85848-47q46"
Oct 31 22:01:50.854453 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:50.854419    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-75fdc85848-47q46"]
Oct 31 22:01:50.855367 ip-10-0-136-157 kubenswrapper[4460]: W1031 22:01:50.855342    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcccd0fb2_40d7_4dc3_8170_54e310bf1ce5.slice/crio-cc6d9c727b0b6bb4e32633481108294977e78445b5e23ce614f37e6fe4cad91a WatchSource:0}: Error finding container cc6d9c727b0b6bb4e32633481108294977e78445b5e23ce614f37e6fe4cad91a: Status 404 returned error can't find the container with id cc6d9c727b0b6bb4e32633481108294977e78445b5e23ce614f37e6fe4cad91a
Oct 31 22:01:51.075999 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:51.075920    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-75fdc85848-47q46" event={"ID":"cccd0fb2-40d7-4dc3-8170-54e310bf1ce5","Type":"ContainerStarted","Data":"df4ccc56ec55c29174fb3b29cf46586b5b8bed4e369bfba5243a83a2b2a2c397"}
Oct 31 22:01:51.076403 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:51.076037    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-75fdc85848-47q46"
Oct 31 22:01:51.076403 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:51.076063    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-75fdc85848-47q46" event={"ID":"cccd0fb2-40d7-4dc3-8170-54e310bf1ce5","Type":"ContainerStarted","Data":"cc6d9c727b0b6bb4e32633481108294977e78445b5e23ce614f37e6fe4cad91a"}
Oct 31 22:01:51.125250 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:51.125152    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-75fdc85848-47q46" podStartSLOduration=1.125135125 podStartE2EDuration="1.125135125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:01:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:01:51.123028477 +0000 UTC m=+2751.039337912" watchObservedRunningTime="2026-10-31 22:01:51.125135125 +0000 UTC m=+2751.041444571"
Oct 31 22:01:51.455232 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:01:51.455148    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:01:51.980401 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:51.980363    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:52.262586 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:52.262558    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 22:01:53.823632 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:53.823586    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:55.712205 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:55.712158    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:56.367177 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:56.367135    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:56.478545 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:56.478511    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:57.700276 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:57.700231    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:58.616116 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:58.616078    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:59.477997 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:59.477956    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft"]
Oct 31 22:01:59.484014 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:59.483988    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft"
Oct 31 22:01:59.503009 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:59.502911    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft"]
Oct 31 22:01:59.510841 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:59.510819    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-a5affe\"/\"agc-cred-9fdad15e3d87\""
Oct 31 22:01:59.527404 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:59.527371    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:01:59.635923 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:59.635885    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/e42cca82-6412-44dc-8878-fcfbabecf695-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft\" (UID: \"e42cca82-6412-44dc-8878-fcfbabecf695\") " pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft"
Oct 31 22:01:59.636155 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:59.636128    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e42cca82-6412-44dc-8878-fcfbabecf695-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft\" (UID: \"e42cca82-6412-44dc-8878-fcfbabecf695\") " pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft"
Oct 31 22:01:59.636214 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:59.636158    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8b9kf\" (UniqueName: \"kubernetes.io/projected/e42cca82-6412-44dc-8878-fcfbabecf695-kube-api-access-8b9kf\") pod \"payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft\" (UID: \"e42cca82-6412-44dc-8878-fcfbabecf695\") " pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft"
Oct 31 22:01:59.636299 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:59.636285    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e42cca82-6412-44dc-8878-fcfbabecf695-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft\" (UID: \"e42cca82-6412-44dc-8878-fcfbabecf695\") " pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft"
Oct 31 22:01:59.737267 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:59.737174    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/e42cca82-6412-44dc-8878-fcfbabecf695-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft\" (UID: \"e42cca82-6412-44dc-8878-fcfbabecf695\") " pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft"
Oct 31 22:01:59.737440 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:59.737341    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e42cca82-6412-44dc-8878-fcfbabecf695-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft\" (UID: \"e42cca82-6412-44dc-8878-fcfbabecf695\") " pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft"
Oct 31 22:01:59.737440 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:59.737366    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8b9kf\" (UniqueName: \"kubernetes.io/projected/e42cca82-6412-44dc-8878-fcfbabecf695-kube-api-access-8b9kf\") pod \"payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft\" (UID: \"e42cca82-6412-44dc-8878-fcfbabecf695\") " pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft"
Oct 31 22:01:59.737535 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:59.737451    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e42cca82-6412-44dc-8878-fcfbabecf695-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft\" (UID: \"e42cca82-6412-44dc-8878-fcfbabecf695\") " pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft"
Oct 31 22:01:59.738173 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:59.738150    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e42cca82-6412-44dc-8878-fcfbabecf695-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft\" (UID: \"e42cca82-6412-44dc-8878-fcfbabecf695\") " pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft"
Oct 31 22:01:59.738230 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:59.738186    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e42cca82-6412-44dc-8878-fcfbabecf695-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft\" (UID: \"e42cca82-6412-44dc-8878-fcfbabecf695\") " pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft"
Oct 31 22:01:59.739815 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:59.739793    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/e42cca82-6412-44dc-8878-fcfbabecf695-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft\" (UID: \"e42cca82-6412-44dc-8878-fcfbabecf695\") " pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft"
Oct 31 22:01:59.762608 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:59.762575    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8b9kf\" (UniqueName: \"kubernetes.io/projected/e42cca82-6412-44dc-8878-fcfbabecf695-kube-api-access-8b9kf\") pod \"payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft\" (UID: \"e42cca82-6412-44dc-8878-fcfbabecf695\") " pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft"
Oct 31 22:01:59.802504 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:01:59.802473    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft"
Oct 31 22:02:00.108124 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:00.108082    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft"]
Oct 31 22:02:00.108347 ip-10-0-136-157 kubenswrapper[4460]: W1031 22:02:00.108321    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode42cca82_6412_44dc_8878_fcfbabecf695.slice/crio-bf5005cf0d08abc4179685c441d23c1e491e6e92f80da8855745d5a069141b21 WatchSource:0}: Error finding container bf5005cf0d08abc4179685c441d23c1e491e6e92f80da8855745d5a069141b21: Status 404 returned error can't find the container with id bf5005cf0d08abc4179685c441d23c1e491e6e92f80da8855745d5a069141b21
Oct 31 22:02:00.205842 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:00.205801    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft" event={"ID":"e42cca82-6412-44dc-8878-fcfbabecf695","Type":"ContainerStarted","Data":"addb4e54a59ea979656109cf301b689d5f8bc818e1226548414257db16a80065"}
Oct 31 22:02:00.205996 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:00.205871    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft" event={"ID":"e42cca82-6412-44dc-8878-fcfbabecf695","Type":"ContainerStarted","Data":"bf5005cf0d08abc4179685c441d23c1e491e6e92f80da8855745d5a069141b21"}
Oct 31 22:02:00.205996 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:00.205904    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft"
Oct 31 22:02:00.252435 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:00.252383    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft" podStartSLOduration=1.252368798 podStartE2EDuration="1.252368798s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:01:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:02:00.250531528 +0000 UTC m=+2760.166840955" watchObservedRunningTime="2026-10-31 22:02:00.252368798 +0000 UTC m=+2760.168678238"
Oct 31 22:02:00.884142 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:00.884110    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:01.014158 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:01.014120    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:01.241164 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:01.241085    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj"]
Oct 31 22:02:01.423697 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:01.423662    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:01.543607 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:01.543577    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:02.086582 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:02.086549    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-75fdc85848-47q46"
Oct 31 22:02:02.685061 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:02.685019    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:03.115193 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:03.115158    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:03.697532 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:03.697488    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:03.831967 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:03.831866    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:04.769667 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:04.769630    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft"]
Oct 31 22:02:04.888662 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:04.888628    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:05.016416 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:05.016373    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:05.356366 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:05.356326    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:06.242111 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:06.242072    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj" podUID="69ea00be-260e-490a-bafd-e629d23f57cb" containerName="payload-processing" containerID="cri-o://0d2b60c2fb8824387797d0c1f91f7902def6024a62b3362d2b7cd33a3de92be6" gracePeriod=25
Oct 31 22:02:06.521583 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:06.521556    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj"
Oct 31 22:02:06.583444 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:06.583417    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/69ea00be-260e-490a-bafd-e629d23f57cb-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69ea00be-260e-490a-bafd-e629d23f57cb-plugins-config-volume\") pod \"69ea00be-260e-490a-bafd-e629d23f57cb\" (UID: \"69ea00be-260e-490a-bafd-e629d23f57cb\") "
Oct 31 22:02:06.583631 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:06.583485    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/69ea00be-260e-490a-bafd-e629d23f57cb-provider-credentials\" (UniqueName: \"kubernetes.io/projected/69ea00be-260e-490a-bafd-e629d23f57cb-provider-credentials\") pod \"69ea00be-260e-490a-bafd-e629d23f57cb\" (UID: \"69ea00be-260e-490a-bafd-e629d23f57cb\") "
Oct 31 22:02:06.583631 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:06.583516    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/69ea00be-260e-490a-bafd-e629d23f57cb-kube-api-access-8ktzt\" (UniqueName: \"kubernetes.io/projected/69ea00be-260e-490a-bafd-e629d23f57cb-kube-api-access-8ktzt\") pod \"69ea00be-260e-490a-bafd-e629d23f57cb\" (UID: \"69ea00be-260e-490a-bafd-e629d23f57cb\") "
Oct 31 22:02:06.583631 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:06.583543    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/69ea00be-260e-490a-bafd-e629d23f57cb-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/69ea00be-260e-490a-bafd-e629d23f57cb-routing-overlay-volume\") pod \"69ea00be-260e-490a-bafd-e629d23f57cb\" (UID: \"69ea00be-260e-490a-bafd-e629d23f57cb\") "
Oct 31 22:02:06.584087 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:06.584030    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69ea00be-260e-490a-bafd-e629d23f57cb-routing-overlay-volume" pod "69ea00be-260e-490a-bafd-e629d23f57cb" (UID: "69ea00be-260e-490a-bafd-e629d23f57cb"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:02:06.584214 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:06.584089    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69ea00be-260e-490a-bafd-e629d23f57cb-plugins-config-volume" pod "69ea00be-260e-490a-bafd-e629d23f57cb" (UID: "69ea00be-260e-490a-bafd-e629d23f57cb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:02:06.584367 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:06.584311    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69ea00be-260e-490a-bafd-e629d23f57cb-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:02:06.584367 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:06.584342    4460 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/69ea00be-260e-490a-bafd-e629d23f57cb-routing-overlay-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:02:06.585705 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:06.585676    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69ea00be-260e-490a-bafd-e629d23f57cb-provider-credentials" pod "69ea00be-260e-490a-bafd-e629d23f57cb" (UID: "69ea00be-260e-490a-bafd-e629d23f57cb"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:02:06.586011 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:06.585931    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69ea00be-260e-490a-bafd-e629d23f57cb-kube-api-access-8ktzt" pod "69ea00be-260e-490a-bafd-e629d23f57cb" (UID: "69ea00be-260e-490a-bafd-e629d23f57cb"). InnerVolumeSpecName "kube-api-access-8ktzt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:02:06.685853 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:06.685816    4460 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/69ea00be-260e-490a-bafd-e629d23f57cb-provider-credentials\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:02:06.685853 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:06.685846    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ktzt\" (UniqueName: \"kubernetes.io/projected/69ea00be-260e-490a-bafd-e629d23f57cb-kube-api-access-8ktzt\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:02:06.733441 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:06.733410    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:07.264566 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:07.264524    4460 generic.go:356] "Generic (PLEG): container finished" podID="69ea00be-260e-490a-bafd-e629d23f57cb" containerID="0d2b60c2fb8824387797d0c1f91f7902def6024a62b3362d2b7cd33a3de92be6" exitCode=0
Oct 31 22:02:07.265097 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:07.264580    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj"
Oct 31 22:02:07.265097 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:07.264614    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj" event={"ID":"69ea00be-260e-490a-bafd-e629d23f57cb","Type":"ContainerDied","Data":"0d2b60c2fb8824387797d0c1f91f7902def6024a62b3362d2b7cd33a3de92be6"}
Oct 31 22:02:07.265097 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:07.264689    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj" event={"ID":"69ea00be-260e-490a-bafd-e629d23f57cb","Type":"ContainerDied","Data":"c3f5ab30bcce2ccbb06a52d98fdda6cb4cbb8175f87cce7e93b3d322f979831b"}
Oct 31 22:02:07.265097 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:07.264722    4460 scope.go:122] "RemoveContainer" containerID="0d2b60c2fb8824387797d0c1f91f7902def6024a62b3362d2b7cd33a3de92be6"
Oct 31 22:02:07.276700 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:07.276677    4460 scope.go:122] "RemoveContainer" containerID="0d2b60c2fb8824387797d0c1f91f7902def6024a62b3362d2b7cd33a3de92be6"
Oct 31 22:02:07.277048 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:02:07.277017    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d2b60c2fb8824387797d0c1f91f7902def6024a62b3362d2b7cd33a3de92be6\": container with ID starting with 0d2b60c2fb8824387797d0c1f91f7902def6024a62b3362d2b7cd33a3de92be6 not found: ID does not exist" containerID="0d2b60c2fb8824387797d0c1f91f7902def6024a62b3362d2b7cd33a3de92be6"
Oct 31 22:02:07.277116 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:07.277059    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d2b60c2fb8824387797d0c1f91f7902def6024a62b3362d2b7cd33a3de92be6"} err="failed to get container status \"0d2b60c2fb8824387797d0c1f91f7902def6024a62b3362d2b7cd33a3de92be6\": rpc error: code = NotFound desc = could not find container \"0d2b60c2fb8824387797d0c1f91f7902def6024a62b3362d2b7cd33a3de92be6\": container with ID starting with 0d2b60c2fb8824387797d0c1f91f7902def6024a62b3362d2b7cd33a3de92be6 not found: ID does not exist"
Oct 31 22:02:07.324731 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:07.324698    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj"]
Oct 31 22:02:07.338545 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:07.338511    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-a-fac043/payload-processing-external-model-e2e-extmod-a-fac043-8599mlflj"]
Oct 31 22:02:07.915583 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:07.915554    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:08.039429 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:08.039386    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:08.148464 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:08.148430    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:08.237312 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:08.237238    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="69ea00be-260e-490a-bafd-e629d23f57cb" path="/var/lib/kubelet/pods/69ea00be-260e-490a-bafd-e629d23f57cb/volumes"
Oct 31 22:02:08.430384 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:08.430350    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:09.770611 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:09.770579    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft" podUID="e42cca82-6412-44dc-8878-fcfbabecf695" containerName="payload-processing" containerID="cri-o://addb4e54a59ea979656109cf301b689d5f8bc818e1226548414257db16a80065" gracePeriod=25
Oct 31 22:02:10.050093 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:10.050065    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft"
Oct 31 22:02:10.142092 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:10.142064    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e42cca82-6412-44dc-8878-fcfbabecf695-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e42cca82-6412-44dc-8878-fcfbabecf695-routing-overlay-volume\") pod \"e42cca82-6412-44dc-8878-fcfbabecf695\" (UID: \"e42cca82-6412-44dc-8878-fcfbabecf695\") "
Oct 31 22:02:10.142292 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:10.142108    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e42cca82-6412-44dc-8878-fcfbabecf695-provider-credentials\" (UniqueName: \"kubernetes.io/projected/e42cca82-6412-44dc-8878-fcfbabecf695-provider-credentials\") pod \"e42cca82-6412-44dc-8878-fcfbabecf695\" (UID: \"e42cca82-6412-44dc-8878-fcfbabecf695\") "
Oct 31 22:02:10.142292 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:10.142130    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e42cca82-6412-44dc-8878-fcfbabecf695-kube-api-access-8b9kf\" (UniqueName: \"kubernetes.io/projected/e42cca82-6412-44dc-8878-fcfbabecf695-kube-api-access-8b9kf\") pod \"e42cca82-6412-44dc-8878-fcfbabecf695\" (UID: \"e42cca82-6412-44dc-8878-fcfbabecf695\") "
Oct 31 22:02:10.142529 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:10.142504    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e42cca82-6412-44dc-8878-fcfbabecf695-routing-overlay-volume" pod "e42cca82-6412-44dc-8878-fcfbabecf695" (UID: "e42cca82-6412-44dc-8878-fcfbabecf695"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:02:10.142618 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:10.142511    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e42cca82-6412-44dc-8878-fcfbabecf695-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e42cca82-6412-44dc-8878-fcfbabecf695-plugins-config-volume\") pod \"e42cca82-6412-44dc-8878-fcfbabecf695\" (UID: \"e42cca82-6412-44dc-8878-fcfbabecf695\") "
Oct 31 22:02:10.142886 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:10.142856    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e42cca82-6412-44dc-8878-fcfbabecf695-plugins-config-volume" pod "e42cca82-6412-44dc-8878-fcfbabecf695" (UID: "e42cca82-6412-44dc-8878-fcfbabecf695"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:02:10.143078 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:10.143029    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:10.143341 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:10.143324    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e42cca82-6412-44dc-8878-fcfbabecf695-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:02:10.143382 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:10.143344    4460 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e42cca82-6412-44dc-8878-fcfbabecf695-routing-overlay-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:02:10.144678 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:10.144650    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e42cca82-6412-44dc-8878-fcfbabecf695-provider-credentials" pod "e42cca82-6412-44dc-8878-fcfbabecf695" (UID: "e42cca82-6412-44dc-8878-fcfbabecf695"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:02:10.144832 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:10.144807    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e42cca82-6412-44dc-8878-fcfbabecf695-kube-api-access-8b9kf" pod "e42cca82-6412-44dc-8878-fcfbabecf695" (UID: "e42cca82-6412-44dc-8878-fcfbabecf695"). InnerVolumeSpecName "kube-api-access-8b9kf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:02:10.245345 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:10.245312    4460 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/e42cca82-6412-44dc-8878-fcfbabecf695-provider-credentials\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:02:10.245345 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:10.245341    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8b9kf\" (UniqueName: \"kubernetes.io/projected/e42cca82-6412-44dc-8878-fcfbabecf695-kube-api-access-8b9kf\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:02:10.287310 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:10.287235    4460 generic.go:356] "Generic (PLEG): container finished" podID="e42cca82-6412-44dc-8878-fcfbabecf695" containerID="addb4e54a59ea979656109cf301b689d5f8bc818e1226548414257db16a80065" exitCode=0
Oct 31 22:02:10.287310 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:10.287280    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft"
Oct 31 22:02:10.287500 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:10.287310    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft" event={"ID":"e42cca82-6412-44dc-8878-fcfbabecf695","Type":"ContainerDied","Data":"addb4e54a59ea979656109cf301b689d5f8bc818e1226548414257db16a80065"}
Oct 31 22:02:10.287500 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:10.287363    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft" event={"ID":"e42cca82-6412-44dc-8878-fcfbabecf695","Type":"ContainerDied","Data":"bf5005cf0d08abc4179685c441d23c1e491e6e92f80da8855745d5a069141b21"}
Oct 31 22:02:10.287500 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:10.287388    4460 scope.go:122] "RemoveContainer" containerID="addb4e54a59ea979656109cf301b689d5f8bc818e1226548414257db16a80065"
Oct 31 22:02:10.298450 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:10.298428    4460 scope.go:122] "RemoveContainer" containerID="addb4e54a59ea979656109cf301b689d5f8bc818e1226548414257db16a80065"
Oct 31 22:02:10.298751 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:02:10.298720    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"addb4e54a59ea979656109cf301b689d5f8bc818e1226548414257db16a80065\": container with ID starting with addb4e54a59ea979656109cf301b689d5f8bc818e1226548414257db16a80065 not found: ID does not exist" containerID="addb4e54a59ea979656109cf301b689d5f8bc818e1226548414257db16a80065"
Oct 31 22:02:10.298830 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:10.298763    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"addb4e54a59ea979656109cf301b689d5f8bc818e1226548414257db16a80065"} err="failed to get container status \"addb4e54a59ea979656109cf301b689d5f8bc818e1226548414257db16a80065\": rpc error: code = NotFound desc = could not find container \"addb4e54a59ea979656109cf301b689d5f8bc818e1226548414257db16a80065\": container with ID starting with addb4e54a59ea979656109cf301b689d5f8bc818e1226548414257db16a80065 not found: ID does not exist"
Oct 31 22:02:10.348378 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:10.348348    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft"]
Oct 31 22:02:10.352121 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:10.352099    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-fc47tdbft"]
Oct 31 22:02:11.779620 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:11.779576    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:12.227416 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:12.227336    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e42cca82-6412-44dc-8878-fcfbabecf695" path="/var/lib/kubelet/pods/e42cca82-6412-44dc-8878-fcfbabecf695/volumes"
Oct 31 22:02:13.457649 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:13.457596    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:14.706386 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:14.706353    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:14.815141 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:14.815107    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:15.133467 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:15.133429    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:15.631641 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:15.631603    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:15.753924 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:15.753887    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:17.061718 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:17.061686    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:17.639582 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:17.639543    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:17.648707 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:17.648679    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz"]
Oct 31 22:02:17.748291 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:17.748254    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:18.256459 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:18.256411    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:18.777749 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:18.777716    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:20.321121 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:20.321088    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:21.454091 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:02:21.454052    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:02:21.960121 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:21.960082    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:22.649435 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:22.649389    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz" podUID="db869980-f710-4c15-b810-391a663fe7fd" containerName="payload-processing" containerID="cri-o://17260aea233abd7984de63e270c9a6d99de8797f6d70c872c41ff7a8f5a92165" gracePeriod=25
Oct 31 22:02:22.926252 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:22.926221    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz"
Oct 31 22:02:22.960806 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:22.960774    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db869980-f710-4c15-b810-391a663fe7fd-provider-credentials\" (UniqueName: \"kubernetes.io/projected/db869980-f710-4c15-b810-391a663fe7fd-provider-credentials\") pod \"db869980-f710-4c15-b810-391a663fe7fd\" (UID: \"db869980-f710-4c15-b810-391a663fe7fd\") "
Oct 31 22:02:22.960986 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:22.960883    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db869980-f710-4c15-b810-391a663fe7fd-kube-api-access-dqr8d\" (UniqueName: \"kubernetes.io/projected/db869980-f710-4c15-b810-391a663fe7fd-kube-api-access-dqr8d\") pod \"db869980-f710-4c15-b810-391a663fe7fd\" (UID: \"db869980-f710-4c15-b810-391a663fe7fd\") "
Oct 31 22:02:22.961041 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:22.961011    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/db869980-f710-4c15-b810-391a663fe7fd-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/db869980-f710-4c15-b810-391a663fe7fd-routing-overlay-volume\") pod \"db869980-f710-4c15-b810-391a663fe7fd\" (UID: \"db869980-f710-4c15-b810-391a663fe7fd\") "
Oct 31 22:02:22.961103 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:22.961087    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/db869980-f710-4c15-b810-391a663fe7fd-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db869980-f710-4c15-b810-391a663fe7fd-plugins-config-volume\") pod \"db869980-f710-4c15-b810-391a663fe7fd\" (UID: \"db869980-f710-4c15-b810-391a663fe7fd\") "
Oct 31 22:02:22.961432 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:22.961403    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db869980-f710-4c15-b810-391a663fe7fd-routing-overlay-volume" pod "db869980-f710-4c15-b810-391a663fe7fd" (UID: "db869980-f710-4c15-b810-391a663fe7fd"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:02:22.961540 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:22.961464    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db869980-f710-4c15-b810-391a663fe7fd-plugins-config-volume" pod "db869980-f710-4c15-b810-391a663fe7fd" (UID: "db869980-f710-4c15-b810-391a663fe7fd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:02:22.961691 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:22.961672    4460 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/db869980-f710-4c15-b810-391a663fe7fd-routing-overlay-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:02:22.961762 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:22.961692    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db869980-f710-4c15-b810-391a663fe7fd-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:02:22.962914 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:22.962894    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db869980-f710-4c15-b810-391a663fe7fd-provider-credentials" pod "db869980-f710-4c15-b810-391a663fe7fd" (UID: "db869980-f710-4c15-b810-391a663fe7fd"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:02:22.963038 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:22.962960    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db869980-f710-4c15-b810-391a663fe7fd-kube-api-access-dqr8d" pod "db869980-f710-4c15-b810-391a663fe7fd" (UID: "db869980-f710-4c15-b810-391a663fe7fd"). InnerVolumeSpecName "kube-api-access-dqr8d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:02:23.063067 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:23.063029    4460 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/db869980-f710-4c15-b810-391a663fe7fd-provider-credentials\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:02:23.063067 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:23.063060    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqr8d\" (UniqueName: \"kubernetes.io/projected/db869980-f710-4c15-b810-391a663fe7fd-kube-api-access-dqr8d\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:02:23.420445 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:23.420410    4460 generic.go:356] "Generic (PLEG): container finished" podID="db869980-f710-4c15-b810-391a663fe7fd" containerID="17260aea233abd7984de63e270c9a6d99de8797f6d70c872c41ff7a8f5a92165" exitCode=0
Oct 31 22:02:23.420445 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:23.420453    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz"
Oct 31 22:02:23.420773 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:23.420492    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz" event={"ID":"db869980-f710-4c15-b810-391a663fe7fd","Type":"ContainerDied","Data":"17260aea233abd7984de63e270c9a6d99de8797f6d70c872c41ff7a8f5a92165"}
Oct 31 22:02:23.420773 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:23.420549    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz" event={"ID":"db869980-f710-4c15-b810-391a663fe7fd","Type":"ContainerDied","Data":"425be77c19c7821c536a74241253cde251b3649a856b141b4713d1a5b6ed53c0"}
Oct 31 22:02:23.420773 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:23.420573    4460 scope.go:122] "RemoveContainer" containerID="17260aea233abd7984de63e270c9a6d99de8797f6d70c872c41ff7a8f5a92165"
Oct 31 22:02:23.432022 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:23.432001    4460 scope.go:122] "RemoveContainer" containerID="17260aea233abd7984de63e270c9a6d99de8797f6d70c872c41ff7a8f5a92165"
Oct 31 22:02:23.432289 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:02:23.432267    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17260aea233abd7984de63e270c9a6d99de8797f6d70c872c41ff7a8f5a92165\": container with ID starting with 17260aea233abd7984de63e270c9a6d99de8797f6d70c872c41ff7a8f5a92165 not found: ID does not exist" containerID="17260aea233abd7984de63e270c9a6d99de8797f6d70c872c41ff7a8f5a92165"
Oct 31 22:02:23.432363 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:23.432300    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17260aea233abd7984de63e270c9a6d99de8797f6d70c872c41ff7a8f5a92165"} err="failed to get container status \"17260aea233abd7984de63e270c9a6d99de8797f6d70c872c41ff7a8f5a92165\": rpc error: code = NotFound desc = could not find container \"17260aea233abd7984de63e270c9a6d99de8797f6d70c872c41ff7a8f5a92165\": container with ID starting with 17260aea233abd7984de63e270c9a6d99de8797f6d70c872c41ff7a8f5a92165 not found: ID does not exist"
Oct 31 22:02:23.441022 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:23.440996    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:23.510204 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:23.510137    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz"]
Oct 31 22:02:23.515083 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:23.514725    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-a5affe/payload-processing-external-model-e2e-extmod-b-a5affe-5fbczprpz"]
Oct 31 22:02:24.228272 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:24.228239    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="db869980-f710-4c15-b810-391a663fe7fd" path="/var/lib/kubelet/pods/db869980-f710-4c15-b810-391a663fe7fd/volumes"
Oct 31 22:02:25.033618 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:25.033585    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:25.553785 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:25.553736    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:25.673812 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:25.673749    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:25.785296 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:25.785247    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:26.710699 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:26.710658    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:28.308464 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:28.308431    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:29.509479 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:29.509441    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-75fdc85848-47q46"]
Oct 31 22:02:29.895863 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:29.895822    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:31.351161 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:31.351128    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:31.864966 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:31.864903    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:31.971709 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:31.971668    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:32.373909 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:32.373876    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:32.557082 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:32.557036    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:33.115840 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:33.115799    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:33.366815 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:33.366713    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:34.510337 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:34.510299    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-75fdc85848-47q46" podUID="cccd0fb2-40d7-4dc3-8170-54e310bf1ce5" containerName="payload-processing" containerID="cri-o://df4ccc56ec55c29174fb3b29cf46586b5b8bed4e369bfba5243a83a2b2a2c397" gracePeriod=25
Oct 31 22:02:34.786873 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:34.786784    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:36.373926 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:36.373886    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:37.406460 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:37.406398    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-75fdc85848-47q46"
Oct 31 22:02:37.519091 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:37.519057    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cccd0fb2-40d7-4dc3-8170-54e310bf1ce5-kube-api-access-wc48d\" (UniqueName: \"kubernetes.io/projected/cccd0fb2-40d7-4dc3-8170-54e310bf1ce5-kube-api-access-wc48d\") pod \"cccd0fb2-40d7-4dc3-8170-54e310bf1ce5\" (UID: \"cccd0fb2-40d7-4dc3-8170-54e310bf1ce5\") "
Oct 31 22:02:37.519281 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:37.519234    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cccd0fb2-40d7-4dc3-8170-54e310bf1ce5-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/cccd0fb2-40d7-4dc3-8170-54e310bf1ce5-routing-overlay-volume\") pod \"cccd0fb2-40d7-4dc3-8170-54e310bf1ce5\" (UID: \"cccd0fb2-40d7-4dc3-8170-54e310bf1ce5\") "
Oct 31 22:02:37.519355 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:37.519344    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cccd0fb2-40d7-4dc3-8170-54e310bf1ce5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cccd0fb2-40d7-4dc3-8170-54e310bf1ce5-plugins-config-volume\") pod \"cccd0fb2-40d7-4dc3-8170-54e310bf1ce5\" (UID: \"cccd0fb2-40d7-4dc3-8170-54e310bf1ce5\") "
Oct 31 22:02:37.519423 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:37.519380    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cccd0fb2-40d7-4dc3-8170-54e310bf1ce5-provider-credentials\" (UniqueName: \"kubernetes.io/projected/cccd0fb2-40d7-4dc3-8170-54e310bf1ce5-provider-credentials\") pod \"cccd0fb2-40d7-4dc3-8170-54e310bf1ce5\" (UID: \"cccd0fb2-40d7-4dc3-8170-54e310bf1ce5\") "
Oct 31 22:02:37.519670 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:37.519641    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cccd0fb2-40d7-4dc3-8170-54e310bf1ce5-routing-overlay-volume" pod "cccd0fb2-40d7-4dc3-8170-54e310bf1ce5" (UID: "cccd0fb2-40d7-4dc3-8170-54e310bf1ce5"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:02:37.519778 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:37.519731    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cccd0fb2-40d7-4dc3-8170-54e310bf1ce5-plugins-config-volume" pod "cccd0fb2-40d7-4dc3-8170-54e310bf1ce5" (UID: "cccd0fb2-40d7-4dc3-8170-54e310bf1ce5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:02:37.520026 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:37.520007    4460 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/cccd0fb2-40d7-4dc3-8170-54e310bf1ce5-routing-overlay-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:02:37.520090 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:37.520027    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cccd0fb2-40d7-4dc3-8170-54e310bf1ce5-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:02:37.521308 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:37.521285    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cccd0fb2-40d7-4dc3-8170-54e310bf1ce5-kube-api-access-wc48d" pod "cccd0fb2-40d7-4dc3-8170-54e310bf1ce5" (UID: "cccd0fb2-40d7-4dc3-8170-54e310bf1ce5"). InnerVolumeSpecName "kube-api-access-wc48d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:02:37.521569 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:37.521550    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cccd0fb2-40d7-4dc3-8170-54e310bf1ce5-provider-credentials" pod "cccd0fb2-40d7-4dc3-8170-54e310bf1ce5" (UID: "cccd0fb2-40d7-4dc3-8170-54e310bf1ce5"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:02:37.620359 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:37.620322    4460 generic.go:356] "Generic (PLEG): container finished" podID="cccd0fb2-40d7-4dc3-8170-54e310bf1ce5" containerID="df4ccc56ec55c29174fb3b29cf46586b5b8bed4e369bfba5243a83a2b2a2c397" exitCode=0
Oct 31 22:02:37.620559 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:37.620377    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-75fdc85848-47q46"
Oct 31 22:02:37.620559 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:37.620403    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-75fdc85848-47q46" event={"ID":"cccd0fb2-40d7-4dc3-8170-54e310bf1ce5","Type":"ContainerDied","Data":"df4ccc56ec55c29174fb3b29cf46586b5b8bed4e369bfba5243a83a2b2a2c397"}
Oct 31 22:02:37.620559 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:37.620463    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-75fdc85848-47q46" event={"ID":"cccd0fb2-40d7-4dc3-8170-54e310bf1ce5","Type":"ContainerDied","Data":"cc6d9c727b0b6bb4e32633481108294977e78445b5e23ce614f37e6fe4cad91a"}
Oct 31 22:02:37.620559 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:37.620493    4460 scope.go:122] "RemoveContainer" containerID="df4ccc56ec55c29174fb3b29cf46586b5b8bed4e369bfba5243a83a2b2a2c397"
Oct 31 22:02:37.621194 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:37.621159    4460 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/cccd0fb2-40d7-4dc3-8170-54e310bf1ce5-provider-credentials\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:02:37.621405 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:37.621382    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wc48d\" (UniqueName: \"kubernetes.io/projected/cccd0fb2-40d7-4dc3-8170-54e310bf1ce5-kube-api-access-wc48d\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:02:37.632086 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:37.632066    4460 scope.go:122] "RemoveContainer" containerID="df4ccc56ec55c29174fb3b29cf46586b5b8bed4e369bfba5243a83a2b2a2c397"
Oct 31 22:02:37.632386 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:02:37.632361    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df4ccc56ec55c29174fb3b29cf46586b5b8bed4e369bfba5243a83a2b2a2c397\": container with ID starting with df4ccc56ec55c29174fb3b29cf46586b5b8bed4e369bfba5243a83a2b2a2c397 not found: ID does not exist" containerID="df4ccc56ec55c29174fb3b29cf46586b5b8bed4e369bfba5243a83a2b2a2c397"
Oct 31 22:02:37.632443 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:37.632396    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df4ccc56ec55c29174fb3b29cf46586b5b8bed4e369bfba5243a83a2b2a2c397"} err="failed to get container status \"df4ccc56ec55c29174fb3b29cf46586b5b8bed4e369bfba5243a83a2b2a2c397\": rpc error: code = NotFound desc = could not find container \"df4ccc56ec55c29174fb3b29cf46586b5b8bed4e369bfba5243a83a2b2a2c397\": container with ID starting with df4ccc56ec55c29174fb3b29cf46586b5b8bed4e369bfba5243a83a2b2a2c397 not found: ID does not exist"
Oct 31 22:02:37.678610 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:37.678496    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-75fdc85848-47q46"]
Oct 31 22:02:37.682344 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:37.682316    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-75fdc85848-47q46"]
Oct 31 22:02:37.827775 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:37.827740    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:38.226567 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:38.226534    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cccd0fb2-40d7-4dc3-8170-54e310bf1ce5" path="/var/lib/kubelet/pods/cccd0fb2-40d7-4dc3-8170-54e310bf1ce5/volumes"
Oct 31 22:02:39.417725 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:39.417687    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:40.896906 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:40.896868    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:41.352233 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.352201    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"]
Oct 31 22:02:41.352949 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.352926    4460 cpu_manager.go:414] "containerMap: removing container" podUID="cccd0fb2-40d7-4dc3-8170-54e310bf1ce5" containerName="payload-processing"
Oct 31 22:02:41.353003 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.352958    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cccd0fb2-40d7-4dc3-8170-54e310bf1ce5" containerName="payload-processing"
Oct 31 22:02:41.353003 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.352997    4460 cpu_manager.go:414] "containerMap: removing container" podUID="69ea00be-260e-490a-bafd-e629d23f57cb" containerName="payload-processing"
Oct 31 22:02:41.353003 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.353003    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="69ea00be-260e-490a-bafd-e629d23f57cb" containerName="payload-processing"
Oct 31 22:02:41.353101 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.353019    4460 cpu_manager.go:414] "containerMap: removing container" podUID="e42cca82-6412-44dc-8878-fcfbabecf695" containerName="payload-processing"
Oct 31 22:02:41.353101 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.353026    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e42cca82-6412-44dc-8878-fcfbabecf695" containerName="payload-processing"
Oct 31 22:02:41.353101 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.353038    4460 cpu_manager.go:414] "containerMap: removing container" podUID="db869980-f710-4c15-b810-391a663fe7fd" containerName="payload-processing"
Oct 31 22:02:41.353101 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.353043    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="db869980-f710-4c15-b810-391a663fe7fd" containerName="payload-processing"
Oct 31 22:02:41.353234 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.353175    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="cccd0fb2-40d7-4dc3-8170-54e310bf1ce5" containerName="payload-processing"
Oct 31 22:02:41.353234 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.353190    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="69ea00be-260e-490a-bafd-e629d23f57cb" containerName="payload-processing"
Oct 31 22:02:41.353234 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.353198    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="e42cca82-6412-44dc-8878-fcfbabecf695" containerName="payload-processing"
Oct 31 22:02:41.353234 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.353207    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="db869980-f710-4c15-b810-391a663fe7fd" containerName="payload-processing"
Oct 31 22:02:41.358315 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.358293    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:02:41.392915 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.392882    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"]
Oct 31 22:02:41.474023 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.473983    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-vnvpp\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:02:41.474226 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.474072    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-vnvpp\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:02:41.474226 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.474102    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-vnvpp\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:02:41.474344 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.474256    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd9zj\" (UniqueName: \"kubernetes.io/projected/2412d51b-ffbb-496d-a28e-1a1b3868e98a-kube-api-access-hd9zj\") pod \"maas-default-gateway-openshift-default-58847f9fc5-vnvpp\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:02:41.474344 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.474334    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-vnvpp\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:02:41.474436 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.474379    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-vnvpp\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:02:41.474493 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.474480    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-vnvpp\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:02:41.474540 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.474511    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-vnvpp\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:02:41.474595 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.474544    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-vnvpp\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:02:41.576377 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.576333    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-vnvpp\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:02:41.576582 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.576422    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-vnvpp\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:02:41.576582 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.576456    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-vnvpp\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:02:41.576582 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.576521    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd9zj\" (UniqueName: \"kubernetes.io/projected/2412d51b-ffbb-496d-a28e-1a1b3868e98a-kube-api-access-hd9zj\") pod \"maas-default-gateway-openshift-default-58847f9fc5-vnvpp\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:02:41.576582 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.576572    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-vnvpp\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:02:41.576814 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.576637    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-vnvpp\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:02:41.576814 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.576714    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-vnvpp\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:02:41.576814 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.576742    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-vnvpp\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:02:41.576814 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.576774    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-vnvpp\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:02:41.577058 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.576816    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-vnvpp\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:02:41.577058 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.577048    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-vnvpp\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:02:41.577239 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.577212    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-vnvpp\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:02:41.577309 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.577282    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-vnvpp\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:02:41.577375 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.577357    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-vnvpp\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:02:41.579152 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.579123    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-vnvpp\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:02:41.579281 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.579228    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-vnvpp\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:02:41.602456 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.602388    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd9zj\" (UniqueName: \"kubernetes.io/projected/2412d51b-ffbb-496d-a28e-1a1b3868e98a-kube-api-access-hd9zj\") pod \"maas-default-gateway-openshift-default-58847f9fc5-vnvpp\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:02:41.602815 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.602788    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-vnvpp\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:02:41.677524 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:41.677490    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:02:42.003032 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:42.003001    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"]
Oct 31 22:02:42.004034 ip-10-0-136-157 kubenswrapper[4460]: W1031 22:02:42.004010    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2412d51b_ffbb_496d_a28e_1a1b3868e98a.slice/crio-51f497beb8a5fb4bf775e23cd589d0148b45483d00ad7ad813259be32a91a295 WatchSource:0}: Error finding container 51f497beb8a5fb4bf775e23cd589d0148b45483d00ad7ad813259be32a91a295: Status 404 returned error can't find the container with id 51f497beb8a5fb4bf775e23cd589d0148b45483d00ad7ad813259be32a91a295
Oct 31 22:02:42.006661 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:42.006626    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:02:42.006781 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:42.006710    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:02:42.006781 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:42.006752    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:02:42.433427 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:42.433384    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:42.658101 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:42.658056    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp" event={"ID":"2412d51b-ffbb-496d-a28e-1a1b3868e98a","Type":"ContainerStarted","Data":"0ccf07b9b9fc6878f75ee257a67a0c15e2fb9830b2a7e9a82ec91a542360eb62"}
Oct 31 22:02:42.658283 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:42.658125    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp" event={"ID":"2412d51b-ffbb-496d-a28e-1a1b3868e98a","Type":"ContainerStarted","Data":"51f497beb8a5fb4bf775e23cd589d0148b45483d00ad7ad813259be32a91a295"}
Oct 31 22:02:42.678352 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:42.678318    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:02:42.708308 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:42.708208    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp" podStartSLOduration=1.7081929009999999 podStartE2EDuration="1.708192901s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:02:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:02:42.705654688 +0000 UTC m=+2802.621964117" watchObservedRunningTime="2026-10-31 22:02:42.708192901 +0000 UTC m=+2802.624502330"
Oct 31 22:02:43.679798 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:43.679761    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-vnvpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.204:15021/healthz/ready\": dial tcp 10.133.0.204:15021: connect: connection refused" start-of-body=
Oct 31 22:02:43.680296 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:43.679833    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp" podUID="2412d51b-ffbb-496d-a28e-1a1b3868e98a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.204:15021/healthz/ready\": dial tcp 10.133.0.204:15021: connect: connection refused"
Oct 31 22:02:44.009935 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:44.009901    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:44.678440 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:44.678405    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-vnvpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.204:15021/healthz/ready\": dial tcp 10.133.0.204:15021: connect: connection refused" start-of-body=
Oct 31 22:02:44.678633 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:44.678469    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp" podUID="2412d51b-ffbb-496d-a28e-1a1b3868e98a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.204:15021/healthz/ready\": dial tcp 10.133.0.204:15021: connect: connection refused"
Oct 31 22:02:45.678473 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:45.678433    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-vnvpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.204:15021/healthz/ready\": dial tcp 10.133.0.204:15021: connect: connection refused" start-of-body=
Oct 31 22:02:45.678905 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:45.678520    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp" podUID="2412d51b-ffbb-496d-a28e-1a1b3868e98a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.204:15021/healthz/ready\": dial tcp 10.133.0.204:15021: connect: connection refused"
Oct 31 22:02:45.710240 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:45.710202    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:46.678250 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:46.678211    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-vnvpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.204:15021/healthz/ready\": dial tcp 10.133.0.204:15021: connect: connection refused" start-of-body=
Oct 31 22:02:46.678467 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:46.678272    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp" podUID="2412d51b-ffbb-496d-a28e-1a1b3868e98a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.204:15021/healthz/ready\": dial tcp 10.133.0.204:15021: connect: connection refused"
Oct 31 22:02:47.347919 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:47.347879    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:47.678165 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:47.678076    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-vnvpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.204:15021/healthz/ready\": dial tcp 10.133.0.204:15021: connect: connection refused" start-of-body=
Oct 31 22:02:47.678165 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:47.678150    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp" podUID="2412d51b-ffbb-496d-a28e-1a1b3868e98a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.204:15021/healthz/ready\": dial tcp 10.133.0.204:15021: connect: connection refused"
Oct 31 22:02:48.678590 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:48.678545    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-vnvpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.204:15021/healthz/ready\": dial tcp 10.133.0.204:15021: connect: connection refused" start-of-body=
Oct 31 22:02:48.679025 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:48.678631    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp" podUID="2412d51b-ffbb-496d-a28e-1a1b3868e98a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.204:15021/healthz/ready\": dial tcp 10.133.0.204:15021: connect: connection refused"
Oct 31 22:02:49.018578 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:49.018542    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:49.156152 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:49.156113    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 22:02:49.360066 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:49.359945    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:49.697509 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:49.697405    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-vnvpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.204:15021/healthz/ready\": dial tcp 10.133.0.204:15021: connect: connection refused" start-of-body=
Oct 31 22:02:49.698169 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:49.698126    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp" podUID="2412d51b-ffbb-496d-a28e-1a1b3868e98a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.204:15021/healthz/ready\": dial tcp 10.133.0.204:15021: connect: connection refused"
Oct 31 22:02:50.695688 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:50.695608    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:51.131757 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:51.131723    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:02:51.132284 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:51.132259    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:02:51.133483 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:51.133458    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:02:51.458401 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:02:51.458298    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:02:52.215579 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:52.215546    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:53.788187 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:53.788148    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:55.388318 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:55.388284    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:56.911783 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:56.911749    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:02:58.402036 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:02:58.401992    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:00.108808 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:00.108768    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:01.821409 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:01.821363    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:03.461700 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:03.461657    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:05.179886 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:05.179851    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:06.782970 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:06.782909    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:06.929963 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:06.924367    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:07.041454 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:07.041361    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:07.409250 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:07.409148    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:08.634493 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:08.634455    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:10.341495 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:10.341438    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:11.436223 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.436189    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"]
Oct 31 22:03:11.443033 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.442998    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:03:11.467793 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.467753    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"]
Oct 31 22:03:11.472323 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.472288    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4b2zj\" (UniqueName: \"kubernetes.io/projected/db9cc38f-cdbc-4ae0-8645-cec895e76033-kube-api-access-4b2zj\") pod \"maas-default-gateway-openshift-default-58847f9fc5-b659t\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:03:11.472515 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.472377    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db9cc38f-cdbc-4ae0-8645-cec895e76033-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-b659t\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:03:11.472588 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.472549    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-b659t\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:03:11.472678 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.472657    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-b659t\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:03:11.472744 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.472691    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-b659t\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:03:11.472807 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.472793    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db9cc38f-cdbc-4ae0-8645-cec895e76033-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-b659t\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:03:11.472886 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.472844    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db9cc38f-cdbc-4ae0-8645-cec895e76033-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-b659t\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:03:11.473050 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.473030    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-b659t\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:03:11.473153 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.473137    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-b659t\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:03:11.483147 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.483110    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"]
Oct 31 22:03:11.489699 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.489668    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:03:11.507284 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.507254    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"]
Oct 31 22:03:11.574469 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.574432    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-96tvd\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:03:11.574653 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.574486    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-b659t\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:03:11.574653 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.574574    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-96tvd\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:03:11.574653 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.574645    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-b659t\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:03:11.574785 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.574689    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b2zj\" (UniqueName: \"kubernetes.io/projected/db9cc38f-cdbc-4ae0-8645-cec895e76033-kube-api-access-4b2zj\") pod \"maas-default-gateway-openshift-default-58847f9fc5-b659t\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:03:11.574785 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.574719    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwlg7\" (UniqueName: \"kubernetes.io/projected/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-kube-api-access-lwlg7\") pod \"maas-default-gateway-openshift-default-58847f9fc5-96tvd\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:03:11.574785 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.574749    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db9cc38f-cdbc-4ae0-8645-cec895e76033-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-b659t\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:03:11.574974 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.574855    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-b659t\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:03:11.574974 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.574906    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-96tvd\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:03:11.574974 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.574897    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-b659t\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:03:11.574974 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.574927    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-b659t\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:03:11.575185 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.575040    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-b659t\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:03:11.575185 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.575076    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-96tvd\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:03:11.575185 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.575112    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-96tvd\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:03:11.575185 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.575126    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-b659t\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:03:11.575185 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.575155    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-96tvd\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:03:11.575397 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.575199    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-b659t\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:03:11.575397 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.575323    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db9cc38f-cdbc-4ae0-8645-cec895e76033-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-b659t\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:03:11.575397 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.575350    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-b659t\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:03:11.575397 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.575378    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-96tvd\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:03:11.575552 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.575403    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-96tvd\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:03:11.575552 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.575434    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db9cc38f-cdbc-4ae0-8645-cec895e76033-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-b659t\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:03:11.575815 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.575795    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db9cc38f-cdbc-4ae0-8645-cec895e76033-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-b659t\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:03:11.576972 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.576926    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-b659t\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:03:11.577580 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.577553    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db9cc38f-cdbc-4ae0-8645-cec895e76033-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-b659t\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:03:11.602221 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.602193    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4b2zj\" (UniqueName: \"kubernetes.io/projected/db9cc38f-cdbc-4ae0-8645-cec895e76033-kube-api-access-4b2zj\") pod \"maas-default-gateway-openshift-default-58847f9fc5-b659t\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:03:11.603603 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.603573    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db9cc38f-cdbc-4ae0-8645-cec895e76033-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-b659t\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:03:11.676558 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.676529    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-96tvd\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:03:11.676733 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.676624    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwlg7\" (UniqueName: \"kubernetes.io/projected/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-kube-api-access-lwlg7\") pod \"maas-default-gateway-openshift-default-58847f9fc5-96tvd\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:03:11.676733 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.676684    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-96tvd\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:03:11.676856 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.676815    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-96tvd\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:03:11.676917 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.676865    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-96tvd\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:03:11.676917 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.676909    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-96tvd\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:03:11.677082 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.677036    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-96tvd\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:03:11.677082 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.677066    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-96tvd\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:03:11.677199 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.677071    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-96tvd\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:03:11.677260 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.677227    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-96tvd\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:03:11.677260 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.677084    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-96tvd\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:03:11.677376 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.677320    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-96tvd\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:03:11.677610 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.677572    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-96tvd\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:03:11.677700 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.677673    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-96tvd\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:03:11.679306 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.679286    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-96tvd\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:03:11.679452 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.679432    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-96tvd\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:03:11.701577 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.701521    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-96tvd\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:03:11.701577 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.701560    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwlg7\" (UniqueName: \"kubernetes.io/projected/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-kube-api-access-lwlg7\") pod \"maas-default-gateway-openshift-default-58847f9fc5-96tvd\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:03:11.758826 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.758799    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:03:11.813585 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:11.811715    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:03:12.030254 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:12.030218    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:12.098799 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:12.098338    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"]
Oct 31 22:03:12.110204 ip-10-0-136-157 kubenswrapper[4460]: W1031 22:03:12.110164    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb9cc38f_cdbc_4ae0_8645_cec895e76033.slice/crio-5df93f653ecd7224c08f7511147ddeb22875093e7fdd5764f5b3e064a89c6e3f WatchSource:0}: Error finding container 5df93f653ecd7224c08f7511147ddeb22875093e7fdd5764f5b3e064a89c6e3f: Status 404 returned error can't find the container with id 5df93f653ecd7224c08f7511147ddeb22875093e7fdd5764f5b3e064a89c6e3f
Oct 31 22:03:12.113916 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:12.113877    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:03:12.114067 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:12.113985    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:03:12.114067 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:12.114038    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:03:12.158818 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:12.158790    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"]
Oct 31 22:03:12.165407 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:12.163925    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:03:12.165407 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:12.164088    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:03:12.165407 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:12.164190    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:03:12.981010 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:12.980910    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t" event={"ID":"db9cc38f-cdbc-4ae0-8645-cec895e76033","Type":"ContainerStarted","Data":"c7a68b6607db9189271bb4488928b5af02890786675d4a05de2b4284ae76d708"}
Oct 31 22:03:12.981460 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:12.981011    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t" event={"ID":"db9cc38f-cdbc-4ae0-8645-cec895e76033","Type":"ContainerStarted","Data":"5df93f653ecd7224c08f7511147ddeb22875093e7fdd5764f5b3e064a89c6e3f"}
Oct 31 22:03:12.982884 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:12.982859    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd" event={"ID":"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3","Type":"ContainerStarted","Data":"15150ad7d6becc4804f2bb1576e4065237440c5ffa1e8c7580109aa49f7289f9"}
Oct 31 22:03:12.983021 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:12.982901    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd" event={"ID":"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3","Type":"ContainerStarted","Data":"dcdbbd32218b06e2de6fbb2e342350a90d681c3b8c0ac66de81171f013ca888d"}
Oct 31 22:03:13.092139 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:13.092085    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t" podStartSLOduration=2.092069442 podStartE2EDuration="2.092069442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:03:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:03:13.038368494 +0000 UTC m=+2832.954677924" watchObservedRunningTime="2026-10-31 22:03:13.092069442 +0000 UTC m=+2833.008378871"
Oct 31 22:03:13.093061 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:13.093030    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd" podStartSLOduration=2.093019297 podStartE2EDuration="2.093019297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:03:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:03:13.089452131 +0000 UTC m=+2833.005761559" watchObservedRunningTime="2026-10-31 22:03:13.093019297 +0000 UTC m=+2833.009328786"
Oct 31 22:03:13.711387 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:13.711356    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:13.762972 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:13.762023    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:03:13.763184 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:13.763136    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-b659t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.205:15021/healthz/ready\": dial tcp 10.133.0.205:15021: connect: connection refused" start-of-body=
Oct 31 22:03:13.763261 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:13.763189    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t" podUID="db9cc38f-cdbc-4ae0-8645-cec895e76033" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.205:15021/healthz/ready\": dial tcp 10.133.0.205:15021: connect: connection refused"
Oct 31 22:03:13.816983 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:13.816643    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:03:13.834970 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:13.831089    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-96tvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.206:15021/healthz/ready\": dial tcp 10.133.0.206:15021: connect: connection refused" start-of-body=
Oct 31 22:03:13.834970 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:13.831161    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd" podUID="2ce9ce6d-57d6-4085-94e5-2f7f600b81b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.206:15021/healthz/ready\": dial tcp 10.133.0.206:15021: connect: connection refused"
Oct 31 22:03:13.884123 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:13.884009    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:14.011616 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:14.011583    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:14.760609 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:14.760569    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-b659t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.205:15021/healthz/ready\": dial tcp 10.133.0.205:15021: connect: connection refused" start-of-body=
Oct 31 22:03:14.760956 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:14.760893    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t" podUID="db9cc38f-cdbc-4ae0-8645-cec895e76033" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.205:15021/healthz/ready\": dial tcp 10.133.0.205:15021: connect: connection refused"
Oct 31 22:03:14.813431 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:14.813397    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-96tvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.206:15021/healthz/ready\": dial tcp 10.133.0.206:15021: connect: connection refused" start-of-body=
Oct 31 22:03:14.813616 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:14.813452    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd" podUID="2ce9ce6d-57d6-4085-94e5-2f7f600b81b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.206:15021/healthz/ready\": dial tcp 10.133.0.206:15021: connect: connection refused"
Oct 31 22:03:15.519706 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:15.519673    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:15.773069 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:15.771586    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-b659t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.205:15021/healthz/ready\": dial tcp 10.133.0.205:15021: connect: connection refused" start-of-body=
Oct 31 22:03:15.773069 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:15.771665    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t" podUID="db9cc38f-cdbc-4ae0-8645-cec895e76033" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.205:15021/healthz/ready\": dial tcp 10.133.0.205:15021: connect: connection refused"
Oct 31 22:03:15.813012 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:15.812980    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-96tvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.206:15021/healthz/ready\": dial tcp 10.133.0.206:15021: connect: connection refused" start-of-body=
Oct 31 22:03:15.813194 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:15.813038    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd" podUID="2ce9ce6d-57d6-4085-94e5-2f7f600b81b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.206:15021/healthz/ready\": dial tcp 10.133.0.206:15021: connect: connection refused"
Oct 31 22:03:16.760455 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:16.760410    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-b659t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.205:15021/healthz/ready\": dial tcp 10.133.0.205:15021: connect: connection refused" start-of-body=
Oct 31 22:03:16.761545 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:16.761083    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t" podUID="db9cc38f-cdbc-4ae0-8645-cec895e76033" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.205:15021/healthz/ready\": dial tcp 10.133.0.205:15021: connect: connection refused"
Oct 31 22:03:16.813273 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:16.813236    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-96tvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.206:15021/healthz/ready\": dial tcp 10.133.0.206:15021: connect: connection refused" start-of-body=
Oct 31 22:03:16.813444 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:16.813296    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd" podUID="2ce9ce6d-57d6-4085-94e5-2f7f600b81b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.206:15021/healthz/ready\": dial tcp 10.133.0.206:15021: connect: connection refused"
Oct 31 22:03:17.407853 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:17.407819    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:17.762453 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:17.762413    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-b659t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.205:15021/healthz/ready\": dial tcp 10.133.0.205:15021: connect: connection refused" start-of-body=
Oct 31 22:03:17.763105 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:17.763070    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t" podUID="db9cc38f-cdbc-4ae0-8645-cec895e76033" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.205:15021/healthz/ready\": dial tcp 10.133.0.205:15021: connect: connection refused"
Oct 31 22:03:17.813212 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:17.813176    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-96tvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.206:15021/healthz/ready\": dial tcp 10.133.0.206:15021: connect: connection refused" start-of-body=
Oct 31 22:03:17.813410 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:17.813237    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd" podUID="2ce9ce6d-57d6-4085-94e5-2f7f600b81b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.206:15021/healthz/ready\": dial tcp 10.133.0.206:15021: connect: connection refused"
Oct 31 22:03:18.759249 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:18.759201    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-b659t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.205:15021/healthz/ready\": dial tcp 10.133.0.205:15021: connect: connection refused" start-of-body=
Oct 31 22:03:18.759475 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:18.759280    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t" podUID="db9cc38f-cdbc-4ae0-8645-cec895e76033" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.205:15021/healthz/ready\": dial tcp 10.133.0.205:15021: connect: connection refused"
Oct 31 22:03:18.813395 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:18.813354    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-96tvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.206:15021/healthz/ready\": dial tcp 10.133.0.206:15021: connect: connection refused" start-of-body=
Oct 31 22:03:18.813904 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:18.813420    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd" podUID="2ce9ce6d-57d6-4085-94e5-2f7f600b81b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.206:15021/healthz/ready\": dial tcp 10.133.0.206:15021: connect: connection refused"
Oct 31 22:03:19.220422 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:19.220384    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:19.760005 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:19.759955    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-b659t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.205:15021/healthz/ready\": dial tcp 10.133.0.205:15021: connect: connection refused" start-of-body=
Oct 31 22:03:19.760193 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:19.760012    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t" podUID="db9cc38f-cdbc-4ae0-8645-cec895e76033" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.205:15021/healthz/ready\": dial tcp 10.133.0.205:15021: connect: connection refused"
Oct 31 22:03:19.813038 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:19.813004    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-96tvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.206:15021/healthz/ready\": dial tcp 10.133.0.206:15021: connect: connection refused" start-of-body=
Oct 31 22:03:19.813181 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:19.813055    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd" podUID="2ce9ce6d-57d6-4085-94e5-2f7f600b81b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.206:15021/healthz/ready\": dial tcp 10.133.0.206:15021: connect: connection refused"
Oct 31 22:03:20.417739 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:20.417702    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:20.558684 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:20.558583    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:20.922175 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:20.922138    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:21.313642 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:21.313594    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:03:21.315090 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:21.315048    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:03:21.316269 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:21.316225    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:03:21.362908 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:21.362289    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:21.478146 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:03:21.469677    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:03:21.494488 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:21.494448    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:21.555175 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:21.555141    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:03:21.555898 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:21.555861    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:03:21.556820 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:21.556794    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:03:22.885759 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:22.885722    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:24.706347 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:24.706310    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:26.380979 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:26.373055    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:27.557397 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:27.557361    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:27.670058 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:27.670025    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:28.185965 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:28.172461    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:29.903044 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:29.903002    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:31.633272 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:31.633235    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:33.306679 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:33.306644    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:33.744859 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:33.744747    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:33.856112 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:33.856071    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:35.231779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:35.231716    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:37.001841 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:37.001809    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:38.692486 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:38.692447    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:40.150151 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:40.150117    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:40.334961 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:40.318685    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:40.479448 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:40.479360    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:41.899803 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:41.899761    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:42.008632 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:42.008583    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:42.439654 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:42.439613    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:44.234038 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:44.233988    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:45.922309 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:45.922265    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:47.598908 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:47.598874    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:48.280419 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:48.280386    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:48.404885 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:48.403034    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:49.291816 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:49.291782    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:49.414082 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:49.412190    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:49.530811 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:49.530758    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:49.642070 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:49.641986    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:50.356594 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:50.356542    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:50.537963 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:50.537477    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:50.822553 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:50.822517    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:50.932669 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:50.932630    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:51.045553 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:51.045515    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:51.173573 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:51.173480    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:51.376821 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:51.376782    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:51.455090 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:03:51.455002    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:03:51.502026 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:51.501903    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:51.958948 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:51.958912    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:52.081503 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:52.081468    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:53.235621 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:53.235585    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:53.363242 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:53.363207    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:53.493177 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:53.493097    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m"]
Oct 31 22:03:53.498160 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:53.498138    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m"
Oct 31 22:03:53.516318 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:53.516290    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m"]
Oct 31 22:03:53.526984 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:53.525559    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-943963\"/\"agc-cred-6eb1c529fc8e\""
Oct 31 22:03:53.526984 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:53.525731    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-943963\"/\"payload-processing-external-model-e2e--1e97e1d3-dockercfg-thwr2\""
Oct 31 22:03:53.526984 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:53.525853    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-943963\"/\"routing-overlay\""
Oct 31 22:03:53.526984 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:53.525893    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-943963\"/\"kube-root-ca.crt\""
Oct 31 22:03:53.527921 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:53.527899    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-943963\"/\"openshift-service-ca.crt\""
Oct 31 22:03:53.527921 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:53.527901    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-943963\"/\"payload-processing-external-model-plugins-e2e-extmod-a-943963\""
Oct 31 22:03:53.704822 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:53.703617    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8k5m\" (UniqueName: \"kubernetes.io/projected/f8aab0bc-b258-4edd-b400-1d2bbdc85705-kube-api-access-g8k5m\") pod \"payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m\" (UID: \"f8aab0bc-b258-4edd-b400-1d2bbdc85705\") " pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m"
Oct 31 22:03:53.704822 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:53.703734    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8aab0bc-b258-4edd-b400-1d2bbdc85705-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m\" (UID: \"f8aab0bc-b258-4edd-b400-1d2bbdc85705\") " pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m"
Oct 31 22:03:53.704822 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:53.703777    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/f8aab0bc-b258-4edd-b400-1d2bbdc85705-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m\" (UID: \"f8aab0bc-b258-4edd-b400-1d2bbdc85705\") " pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m"
Oct 31 22:03:53.704822 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:53.703921    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/f8aab0bc-b258-4edd-b400-1d2bbdc85705-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m\" (UID: \"f8aab0bc-b258-4edd-b400-1d2bbdc85705\") " pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m"
Oct 31 22:03:53.805183 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:53.805128    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8aab0bc-b258-4edd-b400-1d2bbdc85705-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m\" (UID: \"f8aab0bc-b258-4edd-b400-1d2bbdc85705\") " pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m"
Oct 31 22:03:53.805433 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:53.805211    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/f8aab0bc-b258-4edd-b400-1d2bbdc85705-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m\" (UID: \"f8aab0bc-b258-4edd-b400-1d2bbdc85705\") " pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m"
Oct 31 22:03:53.805433 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:53.805376    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/f8aab0bc-b258-4edd-b400-1d2bbdc85705-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m\" (UID: \"f8aab0bc-b258-4edd-b400-1d2bbdc85705\") " pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m"
Oct 31 22:03:53.805579 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:53.805535    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8k5m\" (UniqueName: \"kubernetes.io/projected/f8aab0bc-b258-4edd-b400-1d2bbdc85705-kube-api-access-g8k5m\") pod \"payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m\" (UID: \"f8aab0bc-b258-4edd-b400-1d2bbdc85705\") " pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m"
Oct 31 22:03:53.807463 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:53.806741    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8aab0bc-b258-4edd-b400-1d2bbdc85705-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m\" (UID: \"f8aab0bc-b258-4edd-b400-1d2bbdc85705\") " pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m"
Oct 31 22:03:53.807749 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:53.807728    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/f8aab0bc-b258-4edd-b400-1d2bbdc85705-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m\" (UID: \"f8aab0bc-b258-4edd-b400-1d2bbdc85705\") " pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m"
Oct 31 22:03:53.811640 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:53.811610    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/f8aab0bc-b258-4edd-b400-1d2bbdc85705-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m\" (UID: \"f8aab0bc-b258-4edd-b400-1d2bbdc85705\") " pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m"
Oct 31 22:03:53.830921 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:53.830887    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8k5m\" (UniqueName: \"kubernetes.io/projected/f8aab0bc-b258-4edd-b400-1d2bbdc85705-kube-api-access-g8k5m\") pod \"payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m\" (UID: \"f8aab0bc-b258-4edd-b400-1d2bbdc85705\") " pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m"
Oct 31 22:03:54.115376 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:54.115288    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m"
Oct 31 22:03:54.423597 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:54.423564    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m"]
Oct 31 22:03:54.425371 ip-10-0-136-157 kubenswrapper[4460]: W1031 22:03:54.425348    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8aab0bc_b258_4edd_b400_1d2bbdc85705.slice/crio-8e64e0491c8a17f7ee3108e92c30a2211ed988620ef670ddafaf6de58a6d22d9 WatchSource:0}: Error finding container 8e64e0491c8a17f7ee3108e92c30a2211ed988620ef670ddafaf6de58a6d22d9: Status 404 returned error can't find the container with id 8e64e0491c8a17f7ee3108e92c30a2211ed988620ef670ddafaf6de58a6d22d9
Oct 31 22:03:54.668448 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:54.668401    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m" event={"ID":"f8aab0bc-b258-4edd-b400-1d2bbdc85705","Type":"ContainerStarted","Data":"a4a7e6a14463d6dc307d1f915a6abaf714efa611f68d785bb2e0dba28ef8ed5c"}
Oct 31 22:03:54.668602 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:54.668479    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m"
Oct 31 22:03:54.668602 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:54.668506    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m" event={"ID":"f8aab0bc-b258-4edd-b400-1d2bbdc85705","Type":"ContainerStarted","Data":"8e64e0491c8a17f7ee3108e92c30a2211ed988620ef670ddafaf6de58a6d22d9"}
Oct 31 22:03:54.714315 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:54.714206    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m" podStartSLOduration=1.71418972 podStartE2EDuration="1.71418972s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:03:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:03:54.712772566 +0000 UTC m=+2874.629081995" watchObservedRunningTime="2026-10-31 22:03:54.71418972 +0000 UTC m=+2874.630499149"
Oct 31 22:03:54.944191 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:54.944152    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:56.354421 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:56.354388    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 22:03:56.634157 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:56.634057    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:58.064782 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:58.064733    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:58.190450 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:58.190412    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:58.306185 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:58.306145    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:03:58.420248 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:03:58.420148    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:00.592576 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:00.591864    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh" podUID="869ed893-352c-4f8f-a174-b42afdeb4de7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 22:04:00.619826 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:00.618205    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:02.400469 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:02.400430    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:04.022317 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:04.022276    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:04.503553 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:04.503458    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:04.620260 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:04.620217    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:04.731774 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:04.731742    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:04.929194 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:04.929158    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:05.218834 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:05.218751    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:05.679793 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:05.679760    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m"
Oct 31 22:04:05.742913 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:05.742875    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:05.850873 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:05.850832    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:07.293238 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:07.293198    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:07.747124 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:07.747032    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:08.324189 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:08.324155    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:09.381019 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:09.380977    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:11.141316 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:11.141272    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:11.744362 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:11.744326    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:12.918634 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:12.918596    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:14.634369 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:14.634334    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:16.306584 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:16.306540    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:17.931479 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:17.931438    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:19.622319 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:19.622278    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:21.330626 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:21.330581    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:21.454184 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:04:21.454143    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:04:23.003623 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:23.003578    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:23.242950 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:23.242903    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:23.350696 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:23.350656    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:23.578247 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:23.578207    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:24.794924 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:24.794887    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:24.902620 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:24.902578    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:25.041689 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:25.039334    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:26.726428 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:26.726389    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:28.420290 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:28.420257    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:30.237956 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:30.237891    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:32.266999 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:32.266880    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:32.670023 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:32.669729    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-hcd5s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 22:04:32.670023 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:32.669820    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-hcd5s" podUID="c0272849-0715-4e09-8d0b-827116fb35d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 22:04:34.105634 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:34.105595    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:35.787065 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:35.787026    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:37.550148 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:37.550101    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:39.306727 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:39.306691    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:40.986029 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:40.985997    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:42.815209 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:42.815172    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:44.509578 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:44.509526    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:46.095284 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:46.095227    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:47.598735 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:47.598696    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 22:04:47.846755 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:47.846714    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:49.644489 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:49.644444    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:49.756377 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:49.756328    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:51.731734 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:04:51.731652    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:04:51.853794 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:51.853589    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:52.144484 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:52.144439    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-vnvpp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.204:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 22:04:52.144688 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:52.144515    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp" podUID="2412d51b-ffbb-496d-a28e-1a1b3868e98a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.204:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 22:04:53.501422 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:53.501335    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:55.112537 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:55.112495    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:56.707279 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:56.707242    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:04:58.437867 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:04:58.437826    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:00.103544 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:00.103506    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:01.709152 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:01.709116    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:03.426908 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:03.426870    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:05.116711 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:05.116670    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:06.821244 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:06.821211    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:08.505127 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:08.505087    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:10.225506 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:10.225457    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:11.843953 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:11.843902    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:13.503964 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:13.503897    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:15.304033 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:15.303989    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:16.935479 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:16.935441    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:18.548336 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:18.548301    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:20.135161 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:20.135121    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:21.454296 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:05:21.454261    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:05:21.719638 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:21.719550    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:22.007120 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:22.007004    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:22.129619 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:22.129567    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:23.438588 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:23.438556    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:23.576964 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:23.576282    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:25.232493 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:25.232451    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:26.912522 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:26.912486    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:28.502242 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:28.502201    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:30.205751 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:30.205711    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:32.009077 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:32.009039    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:33.623476 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:33.623441    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:35.233628 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:35.233578    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:35.426468 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:35.426425    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 22:05:37.121193 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:37.121152    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:38.824570 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:38.824535    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:40.422633 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:40.422588    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:42.043464 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:42.043425    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:43.712543 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:43.712506    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:45.437907 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:45.437864    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:47.108078 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:47.108036    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:48.683320 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:48.683278    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:49.770632 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:49.770591    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:50.426958 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:50.426911    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:51.453811 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:05:51.453772    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:05:52.083996 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:52.083930    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:53.703804 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:53.703771    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:55.357837 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:55.357804    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:57.123074 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:57.123034    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:05:58.820810 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:05:58.820771    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:00.411231 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:00.411198    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:00.816700 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:00.816670    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ssl9_26069598-5dc4-4c96-b55f-dd2f09884419/ovn-acl-logging/0.log"
Oct 31 22:06:00.824066 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:00.824045    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ssl9_26069598-5dc4-4c96-b55f-dd2f09884419/ovn-acl-logging/0.log"
Oct 31 22:06:02.146800 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:02.146762    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:03.725277 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:03.725231    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:05.386109 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:05.386073    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:07.115178 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:07.115140    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:08.729825 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:08.729792    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:10.461535 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:10.461489    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:12.029716 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:12.029675    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:13.169056 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:13.168955    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:13.726028 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:13.725990    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:15.486691 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:15.486646    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:17.115629 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:17.115587    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:18.812046 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:18.812006    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:20.518335 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:20.518295    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:21.454020 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:06:21.453980    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:06:22.201085 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:22.201054    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:23.793218 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:23.793182    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:25.532693 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:25.532649    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:27.200462 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:27.200426    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:28.775591 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:28.775553    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:30.506124 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:30.506081    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:32.117397 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:32.117358    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:33.810340 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:33.810297    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:35.437684 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:35.437645    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:37.087126 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:37.087092    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:37.094818 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:37.094781    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 22:06:38.887304 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:38.887268    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:40.611791 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:40.611753    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:40.859184 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:40.859142    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:40.970328 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:40.970242    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:42.613552 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:42.613511    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:43.348100 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:43.347462    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr"]
Oct 31 22:06:43.353505 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:43.353471    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr"
Oct 31 22:06:43.371796 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:43.371765    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr"]
Oct 31 22:06:43.380966 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:43.379584    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-696f9a\"/\"payload-processing-external-model-plugins-e2e-extmod-a-696f9a\""
Oct 31 22:06:43.380966 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:43.379648    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-696f9a\"/\"agc-cred-ca74aeda362a\""
Oct 31 22:06:43.380966 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:43.379661    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-696f9a\"/\"kube-root-ca.crt\""
Oct 31 22:06:43.380966 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:43.379589    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-696f9a\"/\"agc-cred-0d8dc58753dc\""
Oct 31 22:06:43.380966 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:43.379712    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-696f9a\"/\"routing-overlay\""
Oct 31 22:06:43.398995 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:43.383618    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-696f9a\"/\"openshift-service-ca.crt\""
Oct 31 22:06:43.398995 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:43.383649    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-696f9a\"/\"payload-processing-external-model-e2e--956cda1a-dockercfg-mccg7\""
Oct 31 22:06:43.449425 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:43.449388    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7584\" (UniqueName: \"kubernetes.io/projected/05775b43-7184-490d-ba6d-5364244cf100-kube-api-access-p7584\") pod \"payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr\" (UID: \"05775b43-7184-490d-ba6d-5364244cf100\") " pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr"
Oct 31 22:06:43.449643 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:43.449587    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05775b43-7184-490d-ba6d-5364244cf100-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr\" (UID: \"05775b43-7184-490d-ba6d-5364244cf100\") " pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr"
Oct 31 22:06:43.449799 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:43.449779    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/05775b43-7184-490d-ba6d-5364244cf100-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr\" (UID: \"05775b43-7184-490d-ba6d-5364244cf100\") " pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr"
Oct 31 22:06:43.449921 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:43.449903    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/05775b43-7184-490d-ba6d-5364244cf100-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr\" (UID: \"05775b43-7184-490d-ba6d-5364244cf100\") " pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr"
Oct 31 22:06:43.551060 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:43.551014    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7584\" (UniqueName: \"kubernetes.io/projected/05775b43-7184-490d-ba6d-5364244cf100-kube-api-access-p7584\") pod \"payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr\" (UID: \"05775b43-7184-490d-ba6d-5364244cf100\") " pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr"
Oct 31 22:06:43.551281 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:43.551165    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05775b43-7184-490d-ba6d-5364244cf100-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr\" (UID: \"05775b43-7184-490d-ba6d-5364244cf100\") " pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr"
Oct 31 22:06:43.551354 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:43.551327    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/05775b43-7184-490d-ba6d-5364244cf100-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr\" (UID: \"05775b43-7184-490d-ba6d-5364244cf100\") " pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr"
Oct 31 22:06:43.551445 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:43.551428    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/05775b43-7184-490d-ba6d-5364244cf100-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr\" (UID: \"05775b43-7184-490d-ba6d-5364244cf100\") " pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr"
Oct 31 22:06:43.551908 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:43.551885    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05775b43-7184-490d-ba6d-5364244cf100-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr\" (UID: \"05775b43-7184-490d-ba6d-5364244cf100\") " pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr"
Oct 31 22:06:43.552034 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:43.551911    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/05775b43-7184-490d-ba6d-5364244cf100-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr\" (UID: \"05775b43-7184-490d-ba6d-5364244cf100\") " pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr"
Oct 31 22:06:43.553996 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:43.553971    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/05775b43-7184-490d-ba6d-5364244cf100-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr\" (UID: \"05775b43-7184-490d-ba6d-5364244cf100\") " pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr"
Oct 31 22:06:43.576476 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:43.576449    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7584\" (UniqueName: \"kubernetes.io/projected/05775b43-7184-490d-ba6d-5364244cf100-kube-api-access-p7584\") pod \"payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr\" (UID: \"05775b43-7184-490d-ba6d-5364244cf100\") " pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr"
Oct 31 22:06:43.672443 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:43.672346    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr"
Oct 31 22:06:43.986718 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:43.986687    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr"]
Oct 31 22:06:43.987656 ip-10-0-136-157 kubenswrapper[4460]: W1031 22:06:43.987631    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05775b43_7184_490d_ba6d_5364244cf100.slice/crio-8cccfb91646a1eab5b968d97c0c8389e6cc9a3537dbbf62ba1d9e388baa8b987 WatchSource:0}: Error finding container 8cccfb91646a1eab5b968d97c0c8389e6cc9a3537dbbf62ba1d9e388baa8b987: Status 404 returned error can't find the container with id 8cccfb91646a1eab5b968d97c0c8389e6cc9a3537dbbf62ba1d9e388baa8b987
Oct 31 22:06:44.220549 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:44.220503    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:44.425802 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:44.425764    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr" event={"ID":"05775b43-7184-490d-ba6d-5364244cf100","Type":"ContainerStarted","Data":"d57df35f365384f3a8ea4c924f520bc3db23e65d8f89480098f3359ca6f6f2f6"}
Oct 31 22:06:44.426024 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:44.425818    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr" event={"ID":"05775b43-7184-490d-ba6d-5364244cf100","Type":"ContainerStarted","Data":"8cccfb91646a1eab5b968d97c0c8389e6cc9a3537dbbf62ba1d9e388baa8b987"}
Oct 31 22:06:44.426024 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:44.425848    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr"
Oct 31 22:06:44.471176 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:44.471127    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr" podStartSLOduration=1.471112092 podStartE2EDuration="1.471112092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:06:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:06:44.470399665 +0000 UTC m=+3044.386709098" watchObservedRunningTime="2026-10-31 22:06:44.471112092 +0000 UTC m=+3044.387421530"
Oct 31 22:06:45.992765 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:45.992722    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:47.714873 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:47.714829    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:49.370121 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:49.370082    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:49.888815 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:49.888779    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:51.003198 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:51.003153    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:51.454080 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:06:51.454047    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:06:52.666537 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:52.666497    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:53.678826 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:53.678791    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:53.788320 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:53.788281    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:54.531756 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:54.531716    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:55.436315 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:55.436284    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr"
Oct 31 22:06:56.312967 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:56.312137    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:57.933333 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:57.933284    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:06:59.628346 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:06:59.628313    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:01.246113 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:01.246075    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:02.905584 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:02.905545    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:04.712639 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:04.712601    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:05.821212 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:05.821174    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m"]
Oct 31 22:07:05.825902 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:05.825874    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:05.951229 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:05.951183    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:06.409593 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:06.409560    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:06.671871 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:06.671780    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:08.419801 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:08.419768    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:10.131494 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:10.131453    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:10.822158 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:10.822117    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m" podUID="f8aab0bc-b258-4edd-b400-1d2bbdc85705" containerName="payload-processing" containerID="cri-o://a4a7e6a14463d6dc307d1f915a6abaf714efa611f68d785bb2e0dba28ef8ed5c" gracePeriod=25
Oct 31 22:07:11.100218 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:11.100190    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m"
Oct 31 22:07:11.134606 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:11.134572    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f8aab0bc-b258-4edd-b400-1d2bbdc85705-kube-api-access-g8k5m\" (UniqueName: \"kubernetes.io/projected/f8aab0bc-b258-4edd-b400-1d2bbdc85705-kube-api-access-g8k5m\") pod \"f8aab0bc-b258-4edd-b400-1d2bbdc85705\" (UID: \"f8aab0bc-b258-4edd-b400-1d2bbdc85705\") "
Oct 31 22:07:11.135087 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:11.134685    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f8aab0bc-b258-4edd-b400-1d2bbdc85705-provider-credentials\" (UniqueName: \"kubernetes.io/projected/f8aab0bc-b258-4edd-b400-1d2bbdc85705-provider-credentials\") pod \"f8aab0bc-b258-4edd-b400-1d2bbdc85705\" (UID: \"f8aab0bc-b258-4edd-b400-1d2bbdc85705\") "
Oct 31 22:07:11.135087 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:11.134746    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f8aab0bc-b258-4edd-b400-1d2bbdc85705-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/f8aab0bc-b258-4edd-b400-1d2bbdc85705-routing-overlay-volume\") pod \"f8aab0bc-b258-4edd-b400-1d2bbdc85705\" (UID: \"f8aab0bc-b258-4edd-b400-1d2bbdc85705\") "
Oct 31 22:07:11.135087 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:11.134845    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f8aab0bc-b258-4edd-b400-1d2bbdc85705-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8aab0bc-b258-4edd-b400-1d2bbdc85705-plugins-config-volume\") pod \"f8aab0bc-b258-4edd-b400-1d2bbdc85705\" (UID: \"f8aab0bc-b258-4edd-b400-1d2bbdc85705\") "
Oct 31 22:07:11.135223 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:11.135194    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8aab0bc-b258-4edd-b400-1d2bbdc85705-routing-overlay-volume" pod "f8aab0bc-b258-4edd-b400-1d2bbdc85705" (UID: "f8aab0bc-b258-4edd-b400-1d2bbdc85705"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:07:11.135261 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:11.135245    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8aab0bc-b258-4edd-b400-1d2bbdc85705-plugins-config-volume" pod "f8aab0bc-b258-4edd-b400-1d2bbdc85705" (UID: "f8aab0bc-b258-4edd-b400-1d2bbdc85705"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:07:11.135492 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:11.135477    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8aab0bc-b258-4edd-b400-1d2bbdc85705-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:07:11.135537 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:11.135495    4460 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/f8aab0bc-b258-4edd-b400-1d2bbdc85705-routing-overlay-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:07:11.136860 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:11.136836    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8aab0bc-b258-4edd-b400-1d2bbdc85705-provider-credentials" pod "f8aab0bc-b258-4edd-b400-1d2bbdc85705" (UID: "f8aab0bc-b258-4edd-b400-1d2bbdc85705"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:07:11.136934 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:11.136852    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8aab0bc-b258-4edd-b400-1d2bbdc85705-kube-api-access-g8k5m" pod "f8aab0bc-b258-4edd-b400-1d2bbdc85705" (UID: "f8aab0bc-b258-4edd-b400-1d2bbdc85705"). InnerVolumeSpecName "kube-api-access-g8k5m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:07:11.237094 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:11.237063    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8k5m\" (UniqueName: \"kubernetes.io/projected/f8aab0bc-b258-4edd-b400-1d2bbdc85705-kube-api-access-g8k5m\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:07:11.237094 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:11.237090    4460 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/f8aab0bc-b258-4edd-b400-1d2bbdc85705-provider-credentials\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:07:11.650542 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:11.650496    4460 generic.go:356] "Generic (PLEG): container finished" podID="f8aab0bc-b258-4edd-b400-1d2bbdc85705" containerID="a4a7e6a14463d6dc307d1f915a6abaf714efa611f68d785bb2e0dba28ef8ed5c" exitCode=0
Oct 31 22:07:11.650728 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:11.650552    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m"
Oct 31 22:07:11.650728 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:11.650574    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m" event={"ID":"f8aab0bc-b258-4edd-b400-1d2bbdc85705","Type":"ContainerDied","Data":"a4a7e6a14463d6dc307d1f915a6abaf714efa611f68d785bb2e0dba28ef8ed5c"}
Oct 31 22:07:11.650728 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:11.650634    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m" event={"ID":"f8aab0bc-b258-4edd-b400-1d2bbdc85705","Type":"ContainerDied","Data":"8e64e0491c8a17f7ee3108e92c30a2211ed988620ef670ddafaf6de58a6d22d9"}
Oct 31 22:07:11.650728 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:11.650658    4460 scope.go:122] "RemoveContainer" containerID="a4a7e6a14463d6dc307d1f915a6abaf714efa611f68d785bb2e0dba28ef8ed5c"
Oct 31 22:07:11.673310 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:11.673279    4460 scope.go:122] "RemoveContainer" containerID="a4a7e6a14463d6dc307d1f915a6abaf714efa611f68d785bb2e0dba28ef8ed5c"
Oct 31 22:07:11.673681 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:07:11.673640    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4a7e6a14463d6dc307d1f915a6abaf714efa611f68d785bb2e0dba28ef8ed5c\": container with ID starting with a4a7e6a14463d6dc307d1f915a6abaf714efa611f68d785bb2e0dba28ef8ed5c not found: ID does not exist" containerID="a4a7e6a14463d6dc307d1f915a6abaf714efa611f68d785bb2e0dba28ef8ed5c"
Oct 31 22:07:11.673756 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:11.673686    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4a7e6a14463d6dc307d1f915a6abaf714efa611f68d785bb2e0dba28ef8ed5c"} err="failed to get container status \"a4a7e6a14463d6dc307d1f915a6abaf714efa611f68d785bb2e0dba28ef8ed5c\": rpc error: code = NotFound desc = could not find container \"a4a7e6a14463d6dc307d1f915a6abaf714efa611f68d785bb2e0dba28ef8ed5c\": container with ID starting with a4a7e6a14463d6dc307d1f915a6abaf714efa611f68d785bb2e0dba28ef8ed5c not found: ID does not exist"
Oct 31 22:07:11.709351 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:11.709309    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m"]
Oct 31 22:07:11.719493 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:11.719467    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-a-943963/payload-processing-external-model-e2e-extmod-a-943963-5cb64bk8m"]
Oct 31 22:07:11.738077 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:11.738046    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:12.226598 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:12.226564    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f8aab0bc-b258-4edd-b400-1d2bbdc85705" path="/var/lib/kubelet/pods/f8aab0bc-b258-4edd-b400-1d2bbdc85705/volumes"
Oct 31 22:07:13.043881 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:13.043831    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:13.153861 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:13.153826    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:13.281094 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:13.281054    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:13.442695 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:13.442607    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:13.561157 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:13.561115    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:13.770515 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:13.770470    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:14.434773 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:14.434737    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:14.544701 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:14.544666    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:15.424723 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:15.424684    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:16.082838 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:16.082805    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:17.154261 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:17.154223    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:17.276544 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:17.276503    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:18.828971 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:18.828919    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:20.513165 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:20.513129    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:21.177517 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:21.177478    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:21.303235 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:21.303196    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:21.528960 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:07:21.526298    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:07:22.627412 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:22.627375    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:24.226542 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:24.226502    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5"]
Oct 31 22:07:24.227283 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:24.227266    4460 cpu_manager.go:414] "containerMap: removing container" podUID="f8aab0bc-b258-4edd-b400-1d2bbdc85705" containerName="payload-processing"
Oct 31 22:07:24.227338 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:24.227284    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f8aab0bc-b258-4edd-b400-1d2bbdc85705" containerName="payload-processing"
Oct 31 22:07:24.227402 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:24.227393    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="f8aab0bc-b258-4edd-b400-1d2bbdc85705" containerName="payload-processing"
Oct 31 22:07:24.232873 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:24.232849    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5"
Oct 31 22:07:24.241457 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:24.241426    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5"]
Oct 31 22:07:24.249857 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:24.249830    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-56a6d4\"/\"kube-root-ca.crt\""
Oct 31 22:07:24.250070 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:24.249904    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-56a6d4\"/\"routing-overlay\""
Oct 31 22:07:24.250181 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:24.250030    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-56a6d4\"/\"openshift-service-ca.crt\""
Oct 31 22:07:24.263043 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:24.263016    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-56a6d4\"/\"agc-cred-4379f1a3a622\""
Oct 31 22:07:24.263301 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:24.263041    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-56a6d4\"/\"payload-processing-external-model-plugins-e2e-extmod-b-56a6d4\""
Oct 31 22:07:24.263301 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:24.263115    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-56a6d4\"/\"agc-cred-2c83ed9af069\""
Oct 31 22:07:24.263301 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:24.263276    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-56a6d4\"/\"payload-processing-external-model-e2e--34b55c0e-dockercfg-r5jhw\""
Oct 31 22:07:24.303129 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:24.303081    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/b2ef2765-a48e-4635-81ae-bdf3d41be396-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5\" (UID: \"b2ef2765-a48e-4635-81ae-bdf3d41be396\") " pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5"
Oct 31 22:07:24.303365 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:24.303183    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdwpv\" (UniqueName: \"kubernetes.io/projected/b2ef2765-a48e-4635-81ae-bdf3d41be396-kube-api-access-fdwpv\") pod \"payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5\" (UID: \"b2ef2765-a48e-4635-81ae-bdf3d41be396\") " pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5"
Oct 31 22:07:24.303365 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:24.303268    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2ef2765-a48e-4635-81ae-bdf3d41be396-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5\" (UID: \"b2ef2765-a48e-4635-81ae-bdf3d41be396\") " pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5"
Oct 31 22:07:24.303365 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:24.303325    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/b2ef2765-a48e-4635-81ae-bdf3d41be396-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5\" (UID: \"b2ef2765-a48e-4635-81ae-bdf3d41be396\") " pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5"
Oct 31 22:07:24.328377 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:24.328340    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:24.413756 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:24.411370    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/b2ef2765-a48e-4635-81ae-bdf3d41be396-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5\" (UID: \"b2ef2765-a48e-4635-81ae-bdf3d41be396\") " pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5"
Oct 31 22:07:24.413756 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:24.411492    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdwpv\" (UniqueName: \"kubernetes.io/projected/b2ef2765-a48e-4635-81ae-bdf3d41be396-kube-api-access-fdwpv\") pod \"payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5\" (UID: \"b2ef2765-a48e-4635-81ae-bdf3d41be396\") " pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5"
Oct 31 22:07:24.413756 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:24.411532    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2ef2765-a48e-4635-81ae-bdf3d41be396-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5\" (UID: \"b2ef2765-a48e-4635-81ae-bdf3d41be396\") " pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5"
Oct 31 22:07:24.413756 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:24.411577    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/b2ef2765-a48e-4635-81ae-bdf3d41be396-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5\" (UID: \"b2ef2765-a48e-4635-81ae-bdf3d41be396\") " pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5"
Oct 31 22:07:24.413756 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:24.412655    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/b2ef2765-a48e-4635-81ae-bdf3d41be396-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5\" (UID: \"b2ef2765-a48e-4635-81ae-bdf3d41be396\") " pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5"
Oct 31 22:07:24.418747 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:24.417481    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/b2ef2765-a48e-4635-81ae-bdf3d41be396-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5\" (UID: \"b2ef2765-a48e-4635-81ae-bdf3d41be396\") " pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5"
Oct 31 22:07:24.421441 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:24.421413    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2ef2765-a48e-4635-81ae-bdf3d41be396-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5\" (UID: \"b2ef2765-a48e-4635-81ae-bdf3d41be396\") " pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5"
Oct 31 22:07:24.438142 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:24.438116    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdwpv\" (UniqueName: \"kubernetes.io/projected/b2ef2765-a48e-4635-81ae-bdf3d41be396-kube-api-access-fdwpv\") pod \"payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5\" (UID: \"b2ef2765-a48e-4635-81ae-bdf3d41be396\") " pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5"
Oct 31 22:07:24.554236 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:24.554191    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5"
Oct 31 22:07:24.880006 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:24.879769    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5"]
Oct 31 22:07:25.782764 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:25.782727    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5" event={"ID":"b2ef2765-a48e-4635-81ae-bdf3d41be396","Type":"ContainerStarted","Data":"1f5b3a7344877a8bfd2eda04a5c3f48104425504b36f59754ea4b91f0198ab12"}
Oct 31 22:07:25.783180 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:25.782824    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5"
Oct 31 22:07:25.783180 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:25.782845    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5" event={"ID":"b2ef2765-a48e-4635-81ae-bdf3d41be396","Type":"ContainerStarted","Data":"895ecef6de463a84270c29fca70f619fc4c0d81633755cc4795a0de58ab6f0b0"}
Oct 31 22:07:25.833694 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:25.833637    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5" podStartSLOduration=1.8336234660000001 podStartE2EDuration="1.833623466s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:07:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:07:25.829044967 +0000 UTC m=+3085.745354402" watchObservedRunningTime="2026-10-31 22:07:25.833623466 +0000 UTC m=+3085.749932930"
Oct 31 22:07:26.018240 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:26.018203    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:27.614557 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:27.614514    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:27.757600 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:27.757562    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:28.057502 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:28.057453    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:29.509468 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:29.509430    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:30.562691 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:30.562654    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 22:07:31.199359 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:31.199316    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:33.106582 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:33.106539    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:34.713449 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:34.713413    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:36.387331 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:36.387271    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:36.794328 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:36.794299    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5"
Oct 31 22:07:38.203138 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:38.203102    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:39.132158 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:39.132119    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:39.253485 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:39.253449    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:40.222732 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:40.222691    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:42.082637 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:42.082593    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:43.578315 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:43.578221    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:43.738963 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:43.735569    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:43.864904 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:43.864811    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:44.952228 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:44.952183    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:45.085826 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:45.085787    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:45.549780 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:45.549733    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:45.662237 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:45.662194    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:45.960309 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:45.960217    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"]
Oct 31 22:07:45.965848 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:45.965822    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:07:45.986607 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:45.986580    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"]
Oct 31 22:07:45.994686 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:45.994450    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-710927-openshift-default-dockercfg-rm4nm\""
Oct 31 22:07:46.009219 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.009186    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:46.096965 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.065062    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-workload-certs\") pod \"e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") " pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:07:46.097317 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.097291    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istio-token\") pod \"e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") " pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:07:46.097542 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.097520    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istio-podinfo\") pod \"e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") " pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:07:46.097754 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.097735    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-credential-socket\") pod \"e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") " pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:07:46.097892 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.097877    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istio-data\") pod \"e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") " pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:07:46.098107 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.098089    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-workload-socket\") pod \"e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") " pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:07:46.103029 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.102996    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istiod-ca-cert\") pod \"e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") " pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:07:46.103236 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.103218    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istio-envoy\") pod \"e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") " pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:07:46.114961 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.103384    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsc8d\" (UniqueName: \"kubernetes.io/projected/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-kube-api-access-xsc8d\") pod \"e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") " pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:07:46.172974 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.170457    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:46.217069 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.216932    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-workload-socket\") pod \"e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") " pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:07:46.217069 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.217010    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istiod-ca-cert\") pod \"e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") " pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:07:46.217069 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.217056    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istio-envoy\") pod \"e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") " pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:07:46.217388 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.217121    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsc8d\" (UniqueName: \"kubernetes.io/projected/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-kube-api-access-xsc8d\") pod \"e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") " pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:07:46.217388 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.217280    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-workload-certs\") pod \"e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") " pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:07:46.217388 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.217310    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istio-token\") pod \"e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") " pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:07:46.217562 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.217395    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istio-podinfo\") pod \"e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") " pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:07:46.217562 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.217493    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-credential-socket\") pod \"e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") " pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:07:46.217562 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.217532    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istio-data\") pod \"e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") " pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:07:46.218102 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.218055    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istio-data\") pod \"e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") " pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:07:46.218445 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.218423    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-workload-socket\") pod \"e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") " pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:07:46.220404 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.219108    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istiod-ca-cert\") pod \"e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") " pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:07:46.238116 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.230327    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istio-envoy\") pod \"e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") " pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:07:46.238116 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.233470    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-workload-certs\") pod \"e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") " pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:07:46.238116 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.233728    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-credential-socket\") pod \"e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") " pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:07:46.244969 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.241631    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istio-podinfo\") pod \"e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") " pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:07:46.257480 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.256475    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istio-token\") pod \"e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") " pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:07:46.330468 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.330435    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsc8d\" (UniqueName: \"kubernetes.io/projected/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-kube-api-access-xsc8d\") pod \"e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") " pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:07:46.585242 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.585204    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:07:46.929381 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.929355    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"]
Oct 31 22:07:46.930031 ip-10-0-136-157 kubenswrapper[4460]: W1031 22:07:46.929996    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f79cdb1_ad35_4719_8f2b_591fa5965bc5.slice/crio-138204bcbffa8dcf5f0b79407b4e5e07f2dfb2d05093628ff2fff6312746a4fc WatchSource:0}: Error finding container 138204bcbffa8dcf5f0b79407b4e5e07f2dfb2d05093628ff2fff6312746a4fc: Status 404 returned error can't find the container with id 138204bcbffa8dcf5f0b79407b4e5e07f2dfb2d05093628ff2fff6312746a4fc
Oct 31 22:07:46.932762 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.932729    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:07:46.932841 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.932796    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:07:46.932841 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.932825    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:07:46.988888 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:46.988858    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs" event={"ID":"2f79cdb1-ad35-4719-8f2b-591fa5965bc5","Type":"ContainerStarted","Data":"138204bcbffa8dcf5f0b79407b4e5e07f2dfb2d05093628ff2fff6312746a4fc"}
Oct 31 22:07:47.521374 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:47.521330    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:48.001454 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:48.001369    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs" event={"ID":"2f79cdb1-ad35-4719-8f2b-591fa5965bc5","Type":"ContainerStarted","Data":"0246258b62f25177b9d52adc7f42966be057d5ba2aac2bae130835c6c169f299"}
Oct 31 22:07:48.586147 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:48.586110    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:07:48.593546 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:48.593509    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:07:48.647565 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:48.647513    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs" podStartSLOduration=3.647499598 podStartE2EDuration="3.647499598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:07:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:07:48.067367068 +0000 UTC m=+3107.983676499" watchObservedRunningTime="2026-10-31 22:07:48.647499598 +0000 UTC m=+3108.563809027"
Oct 31 22:07:49.008656 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:49.008576    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:07:49.009577 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:49.009555    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:07:49.213828 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:49.213774    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:50.010359 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:50.010315    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:50.404041 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:50.403915    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:50.510126 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:50.510089    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:51.027394 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:51.027355    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:51.454932 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:07:51.454840    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:07:52.089435 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:52.089394    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:52.201553 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:52.201513    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:52.251927 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:52.251892    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-710927-5bdd596d55-2t8rq"]
Oct 31 22:07:52.256696 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:52.256661    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-710927-5bdd596d55-2t8rq"
Oct 31 22:07:52.276537 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:52.276509    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-710927-5bdd596d55-2t8rq"]
Oct 31 22:07:52.284208 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:52.284177    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-710927-dockercfg-2n4fs\""
Oct 31 22:07:52.318853 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:52.318818    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-710927\""
Oct 31 22:07:52.354022 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:52.323892    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5953786a-52ca-4d85-a60d-a276d6889552-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-710927-5bdd596d55-2t8rq\" (UID: \"5953786a-52ca-4d85-a60d-a276d6889552\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-710927-5bdd596d55-2t8rq"
Oct 31 22:07:52.354022 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:52.324024    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r468v\" (UniqueName: \"kubernetes.io/projected/5953786a-52ca-4d85-a60d-a276d6889552-kube-api-access-r468v\") pod \"payload-pre-processing-e2e-inf-a-710927-5bdd596d55-2t8rq\" (UID: \"5953786a-52ca-4d85-a60d-a276d6889552\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-710927-5bdd596d55-2t8rq"
Oct 31 22:07:52.447964 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:52.444918    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5953786a-52ca-4d85-a60d-a276d6889552-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-710927-5bdd596d55-2t8rq\" (UID: \"5953786a-52ca-4d85-a60d-a276d6889552\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-710927-5bdd596d55-2t8rq"
Oct 31 22:07:52.447964 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:52.445055    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r468v\" (UniqueName: \"kubernetes.io/projected/5953786a-52ca-4d85-a60d-a276d6889552-kube-api-access-r468v\") pod \"payload-pre-processing-e2e-inf-a-710927-5bdd596d55-2t8rq\" (UID: \"5953786a-52ca-4d85-a60d-a276d6889552\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-710927-5bdd596d55-2t8rq"
Oct 31 22:07:52.447964 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:52.446475    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5953786a-52ca-4d85-a60d-a276d6889552-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-710927-5bdd596d55-2t8rq\" (UID: \"5953786a-52ca-4d85-a60d-a276d6889552\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-710927-5bdd596d55-2t8rq"
Oct 31 22:07:52.544960 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:52.539003    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r468v\" (UniqueName: \"kubernetes.io/projected/5953786a-52ca-4d85-a60d-a276d6889552-kube-api-access-r468v\") pod \"payload-pre-processing-e2e-inf-a-710927-5bdd596d55-2t8rq\" (UID: \"5953786a-52ca-4d85-a60d-a276d6889552\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-710927-5bdd596d55-2t8rq"
Oct 31 22:07:52.579968 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:52.579905    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-710927-5bdd596d55-2t8rq"
Oct 31 22:07:52.812876 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:52.812833    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:52.915705 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:52.915669    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-710927-5bdd596d55-2t8rq"]
Oct 31 22:07:52.916257 ip-10-0-136-157 kubenswrapper[4460]: W1031 22:07:52.916227    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5953786a_52ca_4d85_a60d_a276d6889552.slice/crio-d4d99be858a733888ab88dcbd0fd6facb7901bb7300c4007f2bcce3b7049195f WatchSource:0}: Error finding container d4d99be858a733888ab88dcbd0fd6facb7901bb7300c4007f2bcce3b7049195f: Status 404 returned error can't find the container with id d4d99be858a733888ab88dcbd0fd6facb7901bb7300c4007f2bcce3b7049195f
Oct 31 22:07:53.042586 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:53.042545    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-710927-5bdd596d55-2t8rq" event={"ID":"5953786a-52ca-4d85-a60d-a276d6889552","Type":"ContainerStarted","Data":"43a7df353f4b45d7f732e704f2e48a98660f5293fdfa06142293c0ddf1214d8f"}
Oct 31 22:07:53.042758 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:53.042616    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-710927-5bdd596d55-2t8rq" event={"ID":"5953786a-52ca-4d85-a60d-a276d6889552","Type":"ContainerStarted","Data":"d4d99be858a733888ab88dcbd0fd6facb7901bb7300c4007f2bcce3b7049195f"}
Oct 31 22:07:53.042758 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:53.042648    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-710927-5bdd596d55-2t8rq"
Oct 31 22:07:53.051405 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:53.051376    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5"]
Oct 31 22:07:53.094362 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:53.094310    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-710927-5bdd596d55-2t8rq" podStartSLOduration=1.094296839 podStartE2EDuration="1.094296839s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:07:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:07:53.089794997 +0000 UTC m=+3113.006104426" watchObservedRunningTime="2026-10-31 22:07:53.094296839 +0000 UTC m=+3113.010606269"
Oct 31 22:07:53.447504 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:53.447422    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"]
Oct 31 22:07:53.447683 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:53.447649    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh" podUID="1df58c25-2f7d-438b-a45d-0ff511a344ce" containerName="istio-proxy" containerID="cri-o://cb370a98dc6f63c7b854ab2021a8f8e9e5eaac2952b5488b1510437734808b91" gracePeriod=30
Oct 31 22:07:54.487118 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:54.487081    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:56.112387 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:56.112346    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:57.834105 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:57.834063    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:07:58.051666 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.051618    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5" podUID="b2ef2765-a48e-4635-81ae-bdf3d41be396" containerName="payload-processing" containerID="cri-o://1f5b3a7344877a8bfd2eda04a5c3f48104425504b36f59754ea4b91f0198ab12" gracePeriod=25
Oct 31 22:07:58.338914 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.338845    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5"
Oct 31 22:07:58.449424 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.449381    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b2ef2765-a48e-4635-81ae-bdf3d41be396-kube-api-access-fdwpv\" (UniqueName: \"kubernetes.io/projected/b2ef2765-a48e-4635-81ae-bdf3d41be396-kube-api-access-fdwpv\") pod \"b2ef2765-a48e-4635-81ae-bdf3d41be396\" (UID: \"b2ef2765-a48e-4635-81ae-bdf3d41be396\") "
Oct 31 22:07:58.449592 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.449454    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b2ef2765-a48e-4635-81ae-bdf3d41be396-provider-credentials\" (UniqueName: \"kubernetes.io/projected/b2ef2765-a48e-4635-81ae-bdf3d41be396-provider-credentials\") pod \"b2ef2765-a48e-4635-81ae-bdf3d41be396\" (UID: \"b2ef2765-a48e-4635-81ae-bdf3d41be396\") "
Oct 31 22:07:58.449592 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.449588    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b2ef2765-a48e-4635-81ae-bdf3d41be396-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2ef2765-a48e-4635-81ae-bdf3d41be396-plugins-config-volume\") pod \"b2ef2765-a48e-4635-81ae-bdf3d41be396\" (UID: \"b2ef2765-a48e-4635-81ae-bdf3d41be396\") "
Oct 31 22:07:58.449703 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.449621    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b2ef2765-a48e-4635-81ae-bdf3d41be396-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/b2ef2765-a48e-4635-81ae-bdf3d41be396-routing-overlay-volume\") pod \"b2ef2765-a48e-4635-81ae-bdf3d41be396\" (UID: \"b2ef2765-a48e-4635-81ae-bdf3d41be396\") "
Oct 31 22:07:58.450078 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.450048    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2ef2765-a48e-4635-81ae-bdf3d41be396-plugins-config-volume" pod "b2ef2765-a48e-4635-81ae-bdf3d41be396" (UID: "b2ef2765-a48e-4635-81ae-bdf3d41be396"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:07:58.450193 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.450123    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2ef2765-a48e-4635-81ae-bdf3d41be396-routing-overlay-volume" pod "b2ef2765-a48e-4635-81ae-bdf3d41be396" (UID: "b2ef2765-a48e-4635-81ae-bdf3d41be396"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:07:58.450386 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.450366    4460 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/b2ef2765-a48e-4635-81ae-bdf3d41be396-routing-overlay-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:07:58.450424 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.450389    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2ef2765-a48e-4635-81ae-bdf3d41be396-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:07:58.451647 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.451623    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2ef2765-a48e-4635-81ae-bdf3d41be396-provider-credentials" pod "b2ef2765-a48e-4635-81ae-bdf3d41be396" (UID: "b2ef2765-a48e-4635-81ae-bdf3d41be396"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:07:58.451728 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.451666    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2ef2765-a48e-4635-81ae-bdf3d41be396-kube-api-access-fdwpv" pod "b2ef2765-a48e-4635-81ae-bdf3d41be396" (UID: "b2ef2765-a48e-4635-81ae-bdf3d41be396"). InnerVolumeSpecName "kube-api-access-fdwpv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:07:58.552201 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.552156    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdwpv\" (UniqueName: \"kubernetes.io/projected/b2ef2765-a48e-4635-81ae-bdf3d41be396-kube-api-access-fdwpv\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:07:58.552201 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.552203    4460 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/b2ef2765-a48e-4635-81ae-bdf3d41be396-provider-credentials\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:07:58.720815 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.720789    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:07:58.756154 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.756116    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-credential-socket\") pod \"1df58c25-2f7d-438b-a45d-0ff511a344ce\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") "
Oct 31 22:07:58.756319 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.756163    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1df58c25-2f7d-438b-a45d-0ff511a344ce-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1df58c25-2f7d-438b-a45d-0ff511a344ce-istiod-ca-cert\") pod \"1df58c25-2f7d-438b-a45d-0ff511a344ce\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") "
Oct 31 22:07:58.756319 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.756182    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-workload-socket\") pod \"1df58c25-2f7d-438b-a45d-0ff511a344ce\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") "
Oct 31 22:07:58.756319 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.756213    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-istio-data\") pod \"1df58c25-2f7d-438b-a45d-0ff511a344ce\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") "
Oct 31 22:07:58.756319 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.756230    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-istio-envoy\") pod \"1df58c25-2f7d-438b-a45d-0ff511a344ce\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") "
Oct 31 22:07:58.756319 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.756269    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1df58c25-2f7d-438b-a45d-0ff511a344ce-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1df58c25-2f7d-438b-a45d-0ff511a344ce-istio-podinfo\") pod \"1df58c25-2f7d-438b-a45d-0ff511a344ce\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") "
Oct 31 22:07:58.756319 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.756291    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-workload-certs\") pod \"1df58c25-2f7d-438b-a45d-0ff511a344ce\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") "
Oct 31 22:07:58.756673 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.756352    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1df58c25-2f7d-438b-a45d-0ff511a344ce-kube-api-access-jzjkg\" (UniqueName: \"kubernetes.io/projected/1df58c25-2f7d-438b-a45d-0ff511a344ce-kube-api-access-jzjkg\") pod \"1df58c25-2f7d-438b-a45d-0ff511a344ce\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") "
Oct 31 22:07:58.756673 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.756432    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1df58c25-2f7d-438b-a45d-0ff511a344ce-istio-token\" (UniqueName: \"kubernetes.io/projected/1df58c25-2f7d-438b-a45d-0ff511a344ce-istio-token\") pod \"1df58c25-2f7d-438b-a45d-0ff511a344ce\" (UID: \"1df58c25-2f7d-438b-a45d-0ff511a344ce\") "
Oct 31 22:07:58.756673 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.756468    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-credential-socket" pod "1df58c25-2f7d-438b-a45d-0ff511a344ce" (UID: "1df58c25-2f7d-438b-a45d-0ff511a344ce"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:07:58.756673 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.756588    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-workload-socket" pod "1df58c25-2f7d-438b-a45d-0ff511a344ce" (UID: "1df58c25-2f7d-438b-a45d-0ff511a344ce"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:07:58.756673 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.756599    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1df58c25-2f7d-438b-a45d-0ff511a344ce-istiod-ca-cert" pod "1df58c25-2f7d-438b-a45d-0ff511a344ce" (UID: "1df58c25-2f7d-438b-a45d-0ff511a344ce"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:07:58.756975 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.756835    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-workload-certs" pod "1df58c25-2f7d-438b-a45d-0ff511a344ce" (UID: "1df58c25-2f7d-438b-a45d-0ff511a344ce"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:07:58.756975 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.756884    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-istio-data" pod "1df58c25-2f7d-438b-a45d-0ff511a344ce" (UID: "1df58c25-2f7d-438b-a45d-0ff511a344ce"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:07:58.757352 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.757326    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:07:58.757466 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.757357    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1df58c25-2f7d-438b-a45d-0ff511a344ce-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:07:58.757466 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.757373    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:07:58.757466 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.757392    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:07:58.757466 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.757407    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:07:58.758654 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.758629    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1df58c25-2f7d-438b-a45d-0ff511a344ce-istio-podinfo" pod "1df58c25-2f7d-438b-a45d-0ff511a344ce" (UID: "1df58c25-2f7d-438b-a45d-0ff511a344ce"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 22:07:58.758738 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.758667    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-istio-envoy" pod "1df58c25-2f7d-438b-a45d-0ff511a344ce" (UID: "1df58c25-2f7d-438b-a45d-0ff511a344ce"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:07:58.758738 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.758720    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1df58c25-2f7d-438b-a45d-0ff511a344ce-kube-api-access-jzjkg" pod "1df58c25-2f7d-438b-a45d-0ff511a344ce" (UID: "1df58c25-2f7d-438b-a45d-0ff511a344ce"). InnerVolumeSpecName "kube-api-access-jzjkg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:07:58.758820 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.758755    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1df58c25-2f7d-438b-a45d-0ff511a344ce-istio-token" pod "1df58c25-2f7d-438b-a45d-0ff511a344ce" (UID: "1df58c25-2f7d-438b-a45d-0ff511a344ce"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:07:58.859064 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.859023    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1df58c25-2f7d-438b-a45d-0ff511a344ce-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:07:58.859064 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.859057    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1df58c25-2f7d-438b-a45d-0ff511a344ce-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:07:58.859064 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.859067    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzjkg\" (UniqueName: \"kubernetes.io/projected/1df58c25-2f7d-438b-a45d-0ff511a344ce-kube-api-access-jzjkg\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:07:58.859531 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:58.859079    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1df58c25-2f7d-438b-a45d-0ff511a344ce-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:07:59.096083 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:59.096043    4460 generic.go:356] "Generic (PLEG): container finished" podID="1df58c25-2f7d-438b-a45d-0ff511a344ce" containerID="cb370a98dc6f63c7b854ab2021a8f8e9e5eaac2952b5488b1510437734808b91" exitCode=0
Oct 31 22:07:59.096246 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:59.096102    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"
Oct 31 22:07:59.096246 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:59.096136    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh" event={"ID":"1df58c25-2f7d-438b-a45d-0ff511a344ce","Type":"ContainerDied","Data":"cb370a98dc6f63c7b854ab2021a8f8e9e5eaac2952b5488b1510437734808b91"}
Oct 31 22:07:59.096365 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:59.096247    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh" event={"ID":"1df58c25-2f7d-438b-a45d-0ff511a344ce","Type":"ContainerDied","Data":"b5e7b3721355fedf20e7d97df969e980034ca477c6bebf4d04efcb49c6788ac3"}
Oct 31 22:07:59.096365 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:59.096285    4460 scope.go:122] "RemoveContainer" containerID="cb370a98dc6f63c7b854ab2021a8f8e9e5eaac2952b5488b1510437734808b91"
Oct 31 22:07:59.097790 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:59.097766    4460 generic.go:356] "Generic (PLEG): container finished" podID="b2ef2765-a48e-4635-81ae-bdf3d41be396" containerID="1f5b3a7344877a8bfd2eda04a5c3f48104425504b36f59754ea4b91f0198ab12" exitCode=0
Oct 31 22:07:59.097897 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:59.097879    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5"
Oct 31 22:07:59.097935 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:59.097886    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5" event={"ID":"b2ef2765-a48e-4635-81ae-bdf3d41be396","Type":"ContainerDied","Data":"1f5b3a7344877a8bfd2eda04a5c3f48104425504b36f59754ea4b91f0198ab12"}
Oct 31 22:07:59.098029 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:59.097934    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5" event={"ID":"b2ef2765-a48e-4635-81ae-bdf3d41be396","Type":"ContainerDied","Data":"895ecef6de463a84270c29fca70f619fc4c0d81633755cc4795a0de58ab6f0b0"}
Oct 31 22:07:59.108384 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:59.108364    4460 scope.go:122] "RemoveContainer" containerID="cb370a98dc6f63c7b854ab2021a8f8e9e5eaac2952b5488b1510437734808b91"
Oct 31 22:07:59.108716 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:07:59.108689    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb370a98dc6f63c7b854ab2021a8f8e9e5eaac2952b5488b1510437734808b91\": container with ID starting with cb370a98dc6f63c7b854ab2021a8f8e9e5eaac2952b5488b1510437734808b91 not found: ID does not exist" containerID="cb370a98dc6f63c7b854ab2021a8f8e9e5eaac2952b5488b1510437734808b91"
Oct 31 22:07:59.108835 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:59.108726    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb370a98dc6f63c7b854ab2021a8f8e9e5eaac2952b5488b1510437734808b91"} err="failed to get container status \"cb370a98dc6f63c7b854ab2021a8f8e9e5eaac2952b5488b1510437734808b91\": rpc error: code = NotFound desc = could not find container \"cb370a98dc6f63c7b854ab2021a8f8e9e5eaac2952b5488b1510437734808b91\": container with ID starting with cb370a98dc6f63c7b854ab2021a8f8e9e5eaac2952b5488b1510437734808b91 not found: ID does not exist"
Oct 31 22:07:59.108835 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:59.108746    4460 scope.go:122] "RemoveContainer" containerID="1f5b3a7344877a8bfd2eda04a5c3f48104425504b36f59754ea4b91f0198ab12"
Oct 31 22:07:59.119075 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:59.119052    4460 scope.go:122] "RemoveContainer" containerID="1f5b3a7344877a8bfd2eda04a5c3f48104425504b36f59754ea4b91f0198ab12"
Oct 31 22:07:59.119409 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:07:59.119387    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f5b3a7344877a8bfd2eda04a5c3f48104425504b36f59754ea4b91f0198ab12\": container with ID starting with 1f5b3a7344877a8bfd2eda04a5c3f48104425504b36f59754ea4b91f0198ab12 not found: ID does not exist" containerID="1f5b3a7344877a8bfd2eda04a5c3f48104425504b36f59754ea4b91f0198ab12"
Oct 31 22:07:59.119454 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:59.119419    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f5b3a7344877a8bfd2eda04a5c3f48104425504b36f59754ea4b91f0198ab12"} err="failed to get container status \"1f5b3a7344877a8bfd2eda04a5c3f48104425504b36f59754ea4b91f0198ab12\": rpc error: code = NotFound desc = could not find container \"1f5b3a7344877a8bfd2eda04a5c3f48104425504b36f59754ea4b91f0198ab12\": container with ID starting with 1f5b3a7344877a8bfd2eda04a5c3f48104425504b36f59754ea4b91f0198ab12 not found: ID does not exist"
Oct 31 22:07:59.157900 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:59.157865    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5"]
Oct 31 22:07:59.161964 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:59.161919    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-56a6d4/payload-processing-external-model-e2e-extmod-b-56a6d4-6cb74wpg5"]
Oct 31 22:07:59.201351 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:59.201313    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"]
Oct 31 22:07:59.205425 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:59.205394    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-fac043-openshift-default-7bc677c6b9-qtslh"]
Oct 31 22:07:59.541524 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:07:59.541487    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:00.226738 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:00.226704    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1df58c25-2f7d-438b-a45d-0ff511a344ce" path="/var/lib/kubelet/pods/1df58c25-2f7d-438b-a45d-0ff511a344ce/volumes"
Oct 31 22:08:00.227266 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:00.227248    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b2ef2765-a48e-4635-81ae-bdf3d41be396" path="/var/lib/kubelet/pods/b2ef2765-a48e-4635-81ae-bdf3d41be396/volumes"
Oct 31 22:08:01.202913 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:01.202876    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:02.902609 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:02.902571    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:04.058302 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:04.058266    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-710927-5bdd596d55-2t8rq"
Oct 31 22:08:04.904603 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:04.904496    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:07.185527 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:07.185487    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:07.189962 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:07.187321    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-vnvpp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.204:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 22:08:07.189962 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:07.187392    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp" podUID="2412d51b-ffbb-496d-a28e-1a1b3868e98a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.204:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 22:08:07.364114 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:07.364076    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-b659t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.205:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 22:08:07.364413 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:07.364374    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t" podUID="db9cc38f-cdbc-4ae0-8645-cec895e76033" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.205:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 22:08:09.072907 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:09.072866    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:09.350466 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:09.350372    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:09.471370 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:09.471333    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:10.902713 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:10.902670    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:11.012429 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:11.012399    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:11.203607 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:11.203504    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:11.215091 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:11.215055    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq"]
Oct 31 22:08:11.215880 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:11.215864    4460 cpu_manager.go:414] "containerMap: removing container" podUID="b2ef2765-a48e-4635-81ae-bdf3d41be396" containerName="payload-processing"
Oct 31 22:08:11.215934 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:11.215881    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b2ef2765-a48e-4635-81ae-bdf3d41be396" containerName="payload-processing"
Oct 31 22:08:11.215934 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:11.215913    4460 cpu_manager.go:414] "containerMap: removing container" podUID="1df58c25-2f7d-438b-a45d-0ff511a344ce" containerName="istio-proxy"
Oct 31 22:08:11.215934 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:11.215919    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1df58c25-2f7d-438b-a45d-0ff511a344ce" containerName="istio-proxy"
Oct 31 22:08:11.216102 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:11.216055    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="b2ef2765-a48e-4635-81ae-bdf3d41be396" containerName="payload-processing"
Oct 31 22:08:11.216102 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:11.216072    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="1df58c25-2f7d-438b-a45d-0ff511a344ce" containerName="istio-proxy"
Oct 31 22:08:11.221422 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:11.221398    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq"
Oct 31 22:08:11.239616 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:11.239576    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq"]
Oct 31 22:08:11.277963 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:11.258020    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-4a73b1\"/\"agc-cred-c566851d8db1\""
Oct 31 22:08:11.292028 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:11.267153    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-4a73b1\"/\"agc-cred-d10d6425f537\""
Oct 31 22:08:11.292028 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:11.267214    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-4a73b1\"/\"agc-cred-476126e5f762\""
Oct 31 22:08:11.292028 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:11.267269    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-4a73b1\"/\"openshift-service-ca.crt\""
Oct 31 22:08:11.292028 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:11.267386    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-4a73b1\"/\"kube-root-ca.crt\""
Oct 31 22:08:11.292028 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:11.270916    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-4a73b1\"/\"routing-overlay\""
Oct 31 22:08:11.292028 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:11.271797    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-4a73b1\"/\"payload-processing-external-model-e2e--23c252ba-dockercfg-7tkpq\""
Oct 31 22:08:11.292028 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:11.271858    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-4a73b1\"/\"payload-processing-external-model-plugins-e2e-extmod-b-4a73b1\""
Oct 31 22:08:11.324541 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:11.323977    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/7584246a-71a4-4b0c-8549-650ff4c10058-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq\" (UID: \"7584246a-71a4-4b0c-8549-650ff4c10058\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq"
Oct 31 22:08:11.324541 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:11.324108    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7584246a-71a4-4b0c-8549-650ff4c10058-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq\" (UID: \"7584246a-71a4-4b0c-8549-650ff4c10058\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq"
Oct 31 22:08:11.324541 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:11.324140    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lf54m\" (UniqueName: \"kubernetes.io/projected/7584246a-71a4-4b0c-8549-650ff4c10058-kube-api-access-lf54m\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq\" (UID: \"7584246a-71a4-4b0c-8549-650ff4c10058\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq"
Oct 31 22:08:11.324541 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:11.324207    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7584246a-71a4-4b0c-8549-650ff4c10058-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq\" (UID: \"7584246a-71a4-4b0c-8549-650ff4c10058\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq"
Oct 31 22:08:11.425308 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:11.425266    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7584246a-71a4-4b0c-8549-650ff4c10058-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq\" (UID: \"7584246a-71a4-4b0c-8549-650ff4c10058\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq"
Oct 31 22:08:11.425580 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:11.425320    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lf54m\" (UniqueName: \"kubernetes.io/projected/7584246a-71a4-4b0c-8549-650ff4c10058-kube-api-access-lf54m\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq\" (UID: \"7584246a-71a4-4b0c-8549-650ff4c10058\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq"
Oct 31 22:08:11.425580 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:11.425396    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7584246a-71a4-4b0c-8549-650ff4c10058-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq\" (UID: \"7584246a-71a4-4b0c-8549-650ff4c10058\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq"
Oct 31 22:08:11.425723 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:11.425644    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/7584246a-71a4-4b0c-8549-650ff4c10058-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq\" (UID: \"7584246a-71a4-4b0c-8549-650ff4c10058\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq"
Oct 31 22:08:11.426145 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:11.426118    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7584246a-71a4-4b0c-8549-650ff4c10058-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq\" (UID: \"7584246a-71a4-4b0c-8549-650ff4c10058\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq"
Oct 31 22:08:11.426246 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:11.426161    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7584246a-71a4-4b0c-8549-650ff4c10058-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq\" (UID: \"7584246a-71a4-4b0c-8549-650ff4c10058\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq"
Oct 31 22:08:11.428282 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:11.428251    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/7584246a-71a4-4b0c-8549-650ff4c10058-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq\" (UID: \"7584246a-71a4-4b0c-8549-650ff4c10058\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq"
Oct 31 22:08:11.450136 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:11.450104    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lf54m\" (UniqueName: \"kubernetes.io/projected/7584246a-71a4-4b0c-8549-650ff4c10058-kube-api-access-lf54m\") pod \"payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq\" (UID: \"7584246a-71a4-4b0c-8549-650ff4c10058\") " pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq"
Oct 31 22:08:11.540778 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:11.540736    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq"
Oct 31 22:08:11.853487 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:11.853452    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq"]
Oct 31 22:08:11.853905 ip-10-0-136-157 kubenswrapper[4460]: W1031 22:08:11.853880    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7584246a_71a4_4b0c_8549_650ff4c10058.slice/crio-16e14ddc8b1639338e93635fe300aa271e9ffc7813e2a54c6bb65013ad88a304 WatchSource:0}: Error finding container 16e14ddc8b1639338e93635fe300aa271e9ffc7813e2a54c6bb65013ad88a304: Status 404 returned error can't find the container with id 16e14ddc8b1639338e93635fe300aa271e9ffc7813e2a54c6bb65013ad88a304
Oct 31 22:08:12.470051 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:12.470011    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq" event={"ID":"7584246a-71a4-4b0c-8549-650ff4c10058","Type":"ContainerStarted","Data":"6a3eb4330ee454690b301b56ea7631af538fe659eaf517bae9c2b175b79c26b6"}
Oct 31 22:08:12.470445 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:12.470077    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq" event={"ID":"7584246a-71a4-4b0c-8549-650ff4c10058","Type":"ContainerStarted","Data":"16e14ddc8b1639338e93635fe300aa271e9ffc7813e2a54c6bb65013ad88a304"}
Oct 31 22:08:12.470445 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:12.470108    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq"
Oct 31 22:08:12.516086 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:12.516026    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq" podStartSLOduration=1.516011191 podStartE2EDuration="1.516011191s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:08:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:08:12.514223116 +0000 UTC m=+3132.430532547" watchObservedRunningTime="2026-10-31 22:08:12.516011191 +0000 UTC m=+3132.432320620"
Oct 31 22:08:12.954631 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:12.954598    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:14.602500 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:14.602463    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:16.265019 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:16.264926    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:16.421611 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:16.421567    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:16.537327 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:16.537241    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:18.211720 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:18.211679    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:18.826412 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:18.826373    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq"]
Oct 31 22:08:18.830081 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:18.830054    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq"
Oct 31 22:08:19.823662 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:19.823629    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:21.460841 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:08:21.460796    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:08:21.503466 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:21.503424    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:23.207746 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:23.207703    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:23.317604 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:23.317566    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:23.827408 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:23.827356    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq" podUID="7584246a-71a4-4b0c-8549-650ff4c10058" containerName="payload-processing" containerID="cri-o://6a3eb4330ee454690b301b56ea7631af538fe659eaf517bae9c2b175b79c26b6" gracePeriod=25
Oct 31 22:08:24.128026 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:24.127994    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq"
Oct 31 22:08:24.132162 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:24.132138    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7584246a-71a4-4b0c-8549-650ff4c10058-provider-credentials\" (UniqueName: \"kubernetes.io/projected/7584246a-71a4-4b0c-8549-650ff4c10058-provider-credentials\") pod \"7584246a-71a4-4b0c-8549-650ff4c10058\" (UID: \"7584246a-71a4-4b0c-8549-650ff4c10058\") "
Oct 31 22:08:24.132261 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:24.132247    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7584246a-71a4-4b0c-8549-650ff4c10058-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7584246a-71a4-4b0c-8549-650ff4c10058-plugins-config-volume\") pod \"7584246a-71a4-4b0c-8549-650ff4c10058\" (UID: \"7584246a-71a4-4b0c-8549-650ff4c10058\") "
Oct 31 22:08:24.132347 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:24.132322    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7584246a-71a4-4b0c-8549-650ff4c10058-kube-api-access-lf54m\" (UniqueName: \"kubernetes.io/projected/7584246a-71a4-4b0c-8549-650ff4c10058-kube-api-access-lf54m\") pod \"7584246a-71a4-4b0c-8549-650ff4c10058\" (UID: \"7584246a-71a4-4b0c-8549-650ff4c10058\") "
Oct 31 22:08:24.132501 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:24.132468    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7584246a-71a4-4b0c-8549-650ff4c10058-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7584246a-71a4-4b0c-8549-650ff4c10058-routing-overlay-volume\") pod \"7584246a-71a4-4b0c-8549-650ff4c10058\" (UID: \"7584246a-71a4-4b0c-8549-650ff4c10058\") "
Oct 31 22:08:24.132723 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:24.132690    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7584246a-71a4-4b0c-8549-650ff4c10058-plugins-config-volume" pod "7584246a-71a4-4b0c-8549-650ff4c10058" (UID: "7584246a-71a4-4b0c-8549-650ff4c10058"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:08:24.132933 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:24.132907    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7584246a-71a4-4b0c-8549-650ff4c10058-routing-overlay-volume" pod "7584246a-71a4-4b0c-8549-650ff4c10058" (UID: "7584246a-71a4-4b0c-8549-650ff4c10058"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:08:24.133104 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:24.133086    4460 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/7584246a-71a4-4b0c-8549-650ff4c10058-routing-overlay-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:08:24.133185 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:24.133106    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7584246a-71a4-4b0c-8549-650ff4c10058-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:08:24.134512 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:24.134483    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7584246a-71a4-4b0c-8549-650ff4c10058-kube-api-access-lf54m" pod "7584246a-71a4-4b0c-8549-650ff4c10058" (UID: "7584246a-71a4-4b0c-8549-650ff4c10058"). InnerVolumeSpecName "kube-api-access-lf54m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:08:24.134608 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:24.134524    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7584246a-71a4-4b0c-8549-650ff4c10058-provider-credentials" pod "7584246a-71a4-4b0c-8549-650ff4c10058" (UID: "7584246a-71a4-4b0c-8549-650ff4c10058"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:08:24.235305 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:24.235261    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lf54m\" (UniqueName: \"kubernetes.io/projected/7584246a-71a4-4b0c-8549-650ff4c10058-kube-api-access-lf54m\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:08:24.235305 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:24.235300    4460 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/7584246a-71a4-4b0c-8549-650ff4c10058-provider-credentials\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:08:24.659781 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:24.659743    4460 generic.go:356] "Generic (PLEG): container finished" podID="7584246a-71a4-4b0c-8549-650ff4c10058" containerID="6a3eb4330ee454690b301b56ea7631af538fe659eaf517bae9c2b175b79c26b6" exitCode=0
Oct 31 22:08:24.660007 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:24.659788    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq" event={"ID":"7584246a-71a4-4b0c-8549-650ff4c10058","Type":"ContainerDied","Data":"6a3eb4330ee454690b301b56ea7631af538fe659eaf517bae9c2b175b79c26b6"}
Oct 31 22:08:24.660007 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:24.659794    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq"
Oct 31 22:08:24.660007 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:24.659828    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq" event={"ID":"7584246a-71a4-4b0c-8549-650ff4c10058","Type":"ContainerDied","Data":"16e14ddc8b1639338e93635fe300aa271e9ffc7813e2a54c6bb65013ad88a304"}
Oct 31 22:08:24.660007 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:24.659849    4460 scope.go:122] "RemoveContainer" containerID="6a3eb4330ee454690b301b56ea7631af538fe659eaf517bae9c2b175b79c26b6"
Oct 31 22:08:24.670082 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:24.670056    4460 scope.go:122] "RemoveContainer" containerID="6a3eb4330ee454690b301b56ea7631af538fe659eaf517bae9c2b175b79c26b6"
Oct 31 22:08:24.670402 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:08:24.670384    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a3eb4330ee454690b301b56ea7631af538fe659eaf517bae9c2b175b79c26b6\": container with ID starting with 6a3eb4330ee454690b301b56ea7631af538fe659eaf517bae9c2b175b79c26b6 not found: ID does not exist" containerID="6a3eb4330ee454690b301b56ea7631af538fe659eaf517bae9c2b175b79c26b6"
Oct 31 22:08:24.670451 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:24.670409    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a3eb4330ee454690b301b56ea7631af538fe659eaf517bae9c2b175b79c26b6"} err="failed to get container status \"6a3eb4330ee454690b301b56ea7631af538fe659eaf517bae9c2b175b79c26b6\": rpc error: code = NotFound desc = could not find container \"6a3eb4330ee454690b301b56ea7631af538fe659eaf517bae9c2b175b79c26b6\": container with ID starting with 6a3eb4330ee454690b301b56ea7631af538fe659eaf517bae9c2b175b79c26b6 not found: ID does not exist"
Oct 31 22:08:24.718410 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:24.718376    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq"]
Oct 31 22:08:24.731679 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:24.731651    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-4a73b1/payload-processing-external-model-e2e-extmod-b-4a73b1-79cc65shq"]
Oct 31 22:08:24.919260 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:24.919163    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:26.227878 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:26.227836    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7584246a-71a4-4b0c-8549-650ff4c10058" path="/var/lib/kubelet/pods/7584246a-71a4-4b0c-8549-650ff4c10058/volumes"
Oct 31 22:08:26.494137 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:26.494045    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:28.187357 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:28.187315    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:29.842133 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:29.842081    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:31.492125 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:31.492074    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:31.616354 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:31.616317    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:31.724240 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:31.724200    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:33.268293 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:33.268251    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:34.908197 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:34.908161    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:36.531666 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:36.531620    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:37.403701 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:37.403664    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 22:08:38.309935 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:38.309894    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:39.312255 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:39.312220    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:39.431567 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:39.431528    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:39.628476 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:39.628396    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8"]
Oct 31 22:08:39.629224 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:39.629205    4460 cpu_manager.go:414] "containerMap: removing container" podUID="7584246a-71a4-4b0c-8549-650ff4c10058" containerName="payload-processing"
Oct 31 22:08:39.629278 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:39.629225    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7584246a-71a4-4b0c-8549-650ff4c10058" containerName="payload-processing"
Oct 31 22:08:39.629384 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:39.629374    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="7584246a-71a4-4b0c-8549-650ff4c10058" containerName="payload-processing"
Oct 31 22:08:39.633149 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:39.633132    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8"
Oct 31 22:08:39.652576 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:39.652546    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8"]
Oct 31 22:08:39.722123 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:39.722084    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pp8l\" (UniqueName: \"kubernetes.io/projected/e169fadd-0bfe-4254-8604-0c94885f01e9-kube-api-access-8pp8l\") pod \"payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8\" (UID: \"e169fadd-0bfe-4254-8604-0c94885f01e9\") " pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8"
Oct 31 22:08:39.722298 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:39.722271    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e169fadd-0bfe-4254-8604-0c94885f01e9-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8\" (UID: \"e169fadd-0bfe-4254-8604-0c94885f01e9\") " pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8"
Oct 31 22:08:39.722298 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:39.722295    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e169fadd-0bfe-4254-8604-0c94885f01e9-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8\" (UID: \"e169fadd-0bfe-4254-8604-0c94885f01e9\") " pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8"
Oct 31 22:08:39.722491 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:39.722470    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/e169fadd-0bfe-4254-8604-0c94885f01e9-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8\" (UID: \"e169fadd-0bfe-4254-8604-0c94885f01e9\") " pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8"
Oct 31 22:08:39.824239 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:39.824198    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e169fadd-0bfe-4254-8604-0c94885f01e9-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8\" (UID: \"e169fadd-0bfe-4254-8604-0c94885f01e9\") " pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8"
Oct 31 22:08:39.824239 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:39.824252    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e169fadd-0bfe-4254-8604-0c94885f01e9-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8\" (UID: \"e169fadd-0bfe-4254-8604-0c94885f01e9\") " pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8"
Oct 31 22:08:39.824530 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:39.824454    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/e169fadd-0bfe-4254-8604-0c94885f01e9-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8\" (UID: \"e169fadd-0bfe-4254-8604-0c94885f01e9\") " pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8"
Oct 31 22:08:39.824593 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:39.824571    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pp8l\" (UniqueName: \"kubernetes.io/projected/e169fadd-0bfe-4254-8604-0c94885f01e9-kube-api-access-8pp8l\") pod \"payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8\" (UID: \"e169fadd-0bfe-4254-8604-0c94885f01e9\") " pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8"
Oct 31 22:08:39.825174 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:39.825148    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e169fadd-0bfe-4254-8604-0c94885f01e9-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8\" (UID: \"e169fadd-0bfe-4254-8604-0c94885f01e9\") " pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8"
Oct 31 22:08:39.825287 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:39.825236    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e169fadd-0bfe-4254-8604-0c94885f01e9-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8\" (UID: \"e169fadd-0bfe-4254-8604-0c94885f01e9\") " pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8"
Oct 31 22:08:39.827215 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:39.827197    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/e169fadd-0bfe-4254-8604-0c94885f01e9-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8\" (UID: \"e169fadd-0bfe-4254-8604-0c94885f01e9\") " pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8"
Oct 31 22:08:39.852042 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:39.852016    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pp8l\" (UniqueName: \"kubernetes.io/projected/e169fadd-0bfe-4254-8604-0c94885f01e9-kube-api-access-8pp8l\") pod \"payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8\" (UID: \"e169fadd-0bfe-4254-8604-0c94885f01e9\") " pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8"
Oct 31 22:08:39.949453 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:39.948925    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8"
Oct 31 22:08:40.085068 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:40.085033    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:40.554074 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:40.553346    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8"]
Oct 31 22:08:40.795009 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:40.794886    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8" event={"ID":"e169fadd-0bfe-4254-8604-0c94885f01e9","Type":"ContainerStarted","Data":"6b155d4c060a7342c6efa9c66253ac097421c7e73ae1151038946dbace642b67"}
Oct 31 22:08:40.795009 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:40.795001    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8" event={"ID":"e169fadd-0bfe-4254-8604-0c94885f01e9","Type":"ContainerStarted","Data":"e0916632ff45c847f4d76249f7ff48374d3f6eab8a39a9d6912fc9130b8fa862"}
Oct 31 22:08:40.795216 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:40.795138    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8"
Oct 31 22:08:40.841400 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:40.841352    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8" podStartSLOduration=1.841337792 podStartE2EDuration="1.841337792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:08:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:08:40.83949782 +0000 UTC m=+3160.755807254" watchObservedRunningTime="2026-10-31 22:08:40.841337792 +0000 UTC m=+3160.757647223"
Oct 31 22:08:41.826594 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:41.826544    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:43.505361 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:43.505323    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:45.079561 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:45.079520    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:46.821375 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:46.821338    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:48.498664 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:48.498622    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:50.135954 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:50.135903    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:50.287935 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:50.287892    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:51.454524 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:08:51.454476    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:08:51.805308 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:51.805275    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8"
Oct 31 22:08:51.887264 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:51.887222    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr"]
Oct 31 22:08:52.020339 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:52.020292    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:53.709790 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:53.709749    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:55.396551 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:55.396511    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:56.887919 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:56.887878    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr" podUID="05775b43-7184-490d-ba6d-5364244cf100" containerName="payload-processing" containerID="cri-o://d57df35f365384f3a8ea4c924f520bc3db23e65d8f89480098f3359ca6f6f2f6" gracePeriod=25
Oct 31 22:08:57.032872 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:57.032823    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:08:57.209841 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:57.209815    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr"
Oct 31 22:08:57.261133 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:57.261096    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/05775b43-7184-490d-ba6d-5364244cf100-kube-api-access-p7584\" (UniqueName: \"kubernetes.io/projected/05775b43-7184-490d-ba6d-5364244cf100-kube-api-access-p7584\") pod \"05775b43-7184-490d-ba6d-5364244cf100\" (UID: \"05775b43-7184-490d-ba6d-5364244cf100\") "
Oct 31 22:08:57.261300 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:57.261154    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/05775b43-7184-490d-ba6d-5364244cf100-provider-credentials\" (UniqueName: \"kubernetes.io/projected/05775b43-7184-490d-ba6d-5364244cf100-provider-credentials\") pod \"05775b43-7184-490d-ba6d-5364244cf100\" (UID: \"05775b43-7184-490d-ba6d-5364244cf100\") "
Oct 31 22:08:57.261378 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:57.261357    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/05775b43-7184-490d-ba6d-5364244cf100-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/05775b43-7184-490d-ba6d-5364244cf100-routing-overlay-volume\") pod \"05775b43-7184-490d-ba6d-5364244cf100\" (UID: \"05775b43-7184-490d-ba6d-5364244cf100\") "
Oct 31 22:08:57.261451 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:57.261437    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/05775b43-7184-490d-ba6d-5364244cf100-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05775b43-7184-490d-ba6d-5364244cf100-plugins-config-volume\") pod \"05775b43-7184-490d-ba6d-5364244cf100\" (UID: \"05775b43-7184-490d-ba6d-5364244cf100\") "
Oct 31 22:08:57.262346 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:57.261765    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05775b43-7184-490d-ba6d-5364244cf100-routing-overlay-volume" pod "05775b43-7184-490d-ba6d-5364244cf100" (UID: "05775b43-7184-490d-ba6d-5364244cf100"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:08:57.262346 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:57.261832    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05775b43-7184-490d-ba6d-5364244cf100-plugins-config-volume" pod "05775b43-7184-490d-ba6d-5364244cf100" (UID: "05775b43-7184-490d-ba6d-5364244cf100"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:08:57.262346 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:57.262291    4460 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/05775b43-7184-490d-ba6d-5364244cf100-routing-overlay-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:08:57.262346 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:57.262315    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05775b43-7184-490d-ba6d-5364244cf100-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:08:57.264077 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:57.264052    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05775b43-7184-490d-ba6d-5364244cf100-provider-credentials" pod "05775b43-7184-490d-ba6d-5364244cf100" (UID: "05775b43-7184-490d-ba6d-5364244cf100"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:08:57.264206 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:57.264087    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05775b43-7184-490d-ba6d-5364244cf100-kube-api-access-p7584" pod "05775b43-7184-490d-ba6d-5364244cf100" (UID: "05775b43-7184-490d-ba6d-5364244cf100"). InnerVolumeSpecName "kube-api-access-p7584". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:08:57.367652 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:57.367360    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7584\" (UniqueName: \"kubernetes.io/projected/05775b43-7184-490d-ba6d-5364244cf100-kube-api-access-p7584\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:08:57.367652 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:57.367441    4460 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/05775b43-7184-490d-ba6d-5364244cf100-provider-credentials\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:08:57.940734 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:57.940694    4460 generic.go:356] "Generic (PLEG): container finished" podID="05775b43-7184-490d-ba6d-5364244cf100" containerID="d57df35f365384f3a8ea4c924f520bc3db23e65d8f89480098f3359ca6f6f2f6" exitCode=0
Oct 31 22:08:57.941271 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:57.940750    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr"
Oct 31 22:08:57.941271 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:57.940782    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr" event={"ID":"05775b43-7184-490d-ba6d-5364244cf100","Type":"ContainerDied","Data":"d57df35f365384f3a8ea4c924f520bc3db23e65d8f89480098f3359ca6f6f2f6"}
Oct 31 22:08:57.941271 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:57.940855    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr" event={"ID":"05775b43-7184-490d-ba6d-5364244cf100","Type":"ContainerDied","Data":"8cccfb91646a1eab5b968d97c0c8389e6cc9a3537dbbf62ba1d9e388baa8b987"}
Oct 31 22:08:57.941271 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:57.940891    4460 scope.go:122] "RemoveContainer" containerID="d57df35f365384f3a8ea4c924f520bc3db23e65d8f89480098f3359ca6f6f2f6"
Oct 31 22:08:57.952494 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:57.952470    4460 scope.go:122] "RemoveContainer" containerID="d57df35f365384f3a8ea4c924f520bc3db23e65d8f89480098f3359ca6f6f2f6"
Oct 31 22:08:57.952748 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:08:57.952726    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d57df35f365384f3a8ea4c924f520bc3db23e65d8f89480098f3359ca6f6f2f6\": container with ID starting with d57df35f365384f3a8ea4c924f520bc3db23e65d8f89480098f3359ca6f6f2f6 not found: ID does not exist" containerID="d57df35f365384f3a8ea4c924f520bc3db23e65d8f89480098f3359ca6f6f2f6"
Oct 31 22:08:57.952818 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:57.952754    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d57df35f365384f3a8ea4c924f520bc3db23e65d8f89480098f3359ca6f6f2f6"} err="failed to get container status \"d57df35f365384f3a8ea4c924f520bc3db23e65d8f89480098f3359ca6f6f2f6\": rpc error: code = NotFound desc = could not find container \"d57df35f365384f3a8ea4c924f520bc3db23e65d8f89480098f3359ca6f6f2f6\": container with ID starting with d57df35f365384f3a8ea4c924f520bc3db23e65d8f89480098f3359ca6f6f2f6 not found: ID does not exist"
Oct 31 22:08:58.000026 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:57.999993    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr"]
Oct 31 22:08:58.020558 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:58.020528    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-56bds9qrr"]
Oct 31 22:08:58.226767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:58.226684    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="05775b43-7184-490d-ba6d-5364244cf100" path="/var/lib/kubelet/pods/05775b43-7184-490d-ba6d-5364244cf100/volumes"
Oct 31 22:08:58.833738 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:08:58.833700    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:00.503726 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:00.503689    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:02.219644 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:02.219605    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:02.228854 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:02.228823    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8"]
Oct 31 22:09:02.332625 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:02.332576    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:02.449482 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:02.449444    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:02.928288 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:02.928236    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:03.264534 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:03.264495    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:04.013642 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:04.013602    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:05.653302 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:05.653259    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:07.229986 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:07.229930    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8" podUID="e169fadd-0bfe-4254-8604-0c94885f01e9" containerName="payload-processing" containerID="cri-o://6b155d4c060a7342c6efa9c66253ac097421c7e73ae1151038946dbace642b67" gracePeriod=25
Oct 31 22:09:07.307724 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:07.307688    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:07.570709 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:07.570681    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8"
Oct 31 22:09:07.588511 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:07.588478    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e169fadd-0bfe-4254-8604-0c94885f01e9-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e169fadd-0bfe-4254-8604-0c94885f01e9-plugins-config-volume\") pod \"e169fadd-0bfe-4254-8604-0c94885f01e9\" (UID: \"e169fadd-0bfe-4254-8604-0c94885f01e9\") "
Oct 31 22:09:07.588683 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:07.588588    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e169fadd-0bfe-4254-8604-0c94885f01e9-provider-credentials\" (UniqueName: \"kubernetes.io/projected/e169fadd-0bfe-4254-8604-0c94885f01e9-provider-credentials\") pod \"e169fadd-0bfe-4254-8604-0c94885f01e9\" (UID: \"e169fadd-0bfe-4254-8604-0c94885f01e9\") "
Oct 31 22:09:07.588683 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:07.588640    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e169fadd-0bfe-4254-8604-0c94885f01e9-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e169fadd-0bfe-4254-8604-0c94885f01e9-routing-overlay-volume\") pod \"e169fadd-0bfe-4254-8604-0c94885f01e9\" (UID: \"e169fadd-0bfe-4254-8604-0c94885f01e9\") "
Oct 31 22:09:07.588825 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:07.588734    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e169fadd-0bfe-4254-8604-0c94885f01e9-kube-api-access-8pp8l\" (UniqueName: \"kubernetes.io/projected/e169fadd-0bfe-4254-8604-0c94885f01e9-kube-api-access-8pp8l\") pod \"e169fadd-0bfe-4254-8604-0c94885f01e9\" (UID: \"e169fadd-0bfe-4254-8604-0c94885f01e9\") "
Oct 31 22:09:07.588974 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:07.588935    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e169fadd-0bfe-4254-8604-0c94885f01e9-plugins-config-volume" pod "e169fadd-0bfe-4254-8604-0c94885f01e9" (UID: "e169fadd-0bfe-4254-8604-0c94885f01e9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:09:07.589396 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:07.589352    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e169fadd-0bfe-4254-8604-0c94885f01e9-routing-overlay-volume" pod "e169fadd-0bfe-4254-8604-0c94885f01e9" (UID: "e169fadd-0bfe-4254-8604-0c94885f01e9"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:09:07.589521 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:07.589507    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e169fadd-0bfe-4254-8604-0c94885f01e9-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:09:07.589585 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:07.589528    4460 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e169fadd-0bfe-4254-8604-0c94885f01e9-routing-overlay-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:09:07.591010 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:07.590969    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e169fadd-0bfe-4254-8604-0c94885f01e9-provider-credentials" pod "e169fadd-0bfe-4254-8604-0c94885f01e9" (UID: "e169fadd-0bfe-4254-8604-0c94885f01e9"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:09:07.591102 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:07.591012    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e169fadd-0bfe-4254-8604-0c94885f01e9-kube-api-access-8pp8l" pod "e169fadd-0bfe-4254-8604-0c94885f01e9" (UID: "e169fadd-0bfe-4254-8604-0c94885f01e9"). InnerVolumeSpecName "kube-api-access-8pp8l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:09:07.691481 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:07.691442    4460 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/e169fadd-0bfe-4254-8604-0c94885f01e9-provider-credentials\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:09:07.691481 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:07.691475    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pp8l\" (UniqueName: \"kubernetes.io/projected/e169fadd-0bfe-4254-8604-0c94885f01e9-kube-api-access-8pp8l\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:09:08.058509 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:08.058463    4460 generic.go:356] "Generic (PLEG): container finished" podID="e169fadd-0bfe-4254-8604-0c94885f01e9" containerID="6b155d4c060a7342c6efa9c66253ac097421c7e73ae1151038946dbace642b67" exitCode=0
Oct 31 22:09:08.058711 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:08.058516    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8"
Oct 31 22:09:08.058711 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:08.058544    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8" event={"ID":"e169fadd-0bfe-4254-8604-0c94885f01e9","Type":"ContainerDied","Data":"6b155d4c060a7342c6efa9c66253ac097421c7e73ae1151038946dbace642b67"}
Oct 31 22:09:08.058711 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:08.058603    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8" event={"ID":"e169fadd-0bfe-4254-8604-0c94885f01e9","Type":"ContainerDied","Data":"e0916632ff45c847f4d76249f7ff48374d3f6eab8a39a9d6912fc9130b8fa862"}
Oct 31 22:09:08.058711 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:08.058630    4460 scope.go:122] "RemoveContainer" containerID="6b155d4c060a7342c6efa9c66253ac097421c7e73ae1151038946dbace642b67"
Oct 31 22:09:08.070346 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:08.070327    4460 scope.go:122] "RemoveContainer" containerID="6b155d4c060a7342c6efa9c66253ac097421c7e73ae1151038946dbace642b67"
Oct 31 22:09:08.070654 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:09:08.070634    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b155d4c060a7342c6efa9c66253ac097421c7e73ae1151038946dbace642b67\": container with ID starting with 6b155d4c060a7342c6efa9c66253ac097421c7e73ae1151038946dbace642b67 not found: ID does not exist" containerID="6b155d4c060a7342c6efa9c66253ac097421c7e73ae1151038946dbace642b67"
Oct 31 22:09:08.070707 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:08.070664    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b155d4c060a7342c6efa9c66253ac097421c7e73ae1151038946dbace642b67"} err="failed to get container status \"6b155d4c060a7342c6efa9c66253ac097421c7e73ae1151038946dbace642b67\": rpc error: code = NotFound desc = could not find container \"6b155d4c060a7342c6efa9c66253ac097421c7e73ae1151038946dbace642b67\": container with ID starting with 6b155d4c060a7342c6efa9c66253ac097421c7e73ae1151038946dbace642b67 not found: ID does not exist"
Oct 31 22:09:08.116843 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:08.116805    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8"]
Oct 31 22:09:08.130711 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:08.130682    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-a-696f9a/payload-processing-external-model-e2e-extmod-a-696f9a-7bf5r4ll8"]
Oct 31 22:09:08.227610 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:08.227567    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e169fadd-0bfe-4254-8604-0c94885f01e9" path="/var/lib/kubelet/pods/e169fadd-0bfe-4254-8604-0c94885f01e9/volumes"
Oct 31 22:09:08.949672 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:08.949640    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:09.245161 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:09.245060    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:09.371370 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:09.371333    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:09.698655 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:09.698616    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:10.274851 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.273679    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"]
Oct 31 22:09:10.275672 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.275643    4460 cpu_manager.go:414] "containerMap: removing container" podUID="e169fadd-0bfe-4254-8604-0c94885f01e9" containerName="payload-processing"
Oct 31 22:09:10.276998 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.276973    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e169fadd-0bfe-4254-8604-0c94885f01e9" containerName="payload-processing"
Oct 31 22:09:10.277131 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.277036    4460 cpu_manager.go:414] "containerMap: removing container" podUID="05775b43-7184-490d-ba6d-5364244cf100" containerName="payload-processing"
Oct 31 22:09:10.277131 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.277046    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="05775b43-7184-490d-ba6d-5364244cf100" containerName="payload-processing"
Oct 31 22:09:10.277263 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.277249    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="05775b43-7184-490d-ba6d-5364244cf100" containerName="payload-processing"
Oct 31 22:09:10.277333 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.277283    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="e169fadd-0bfe-4254-8604-0c94885f01e9" containerName="payload-processing"
Oct 31 22:09:10.286117 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.284999    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"]
Oct 31 22:09:10.286117 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.285131    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:09:10.303642 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.300572    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:10.303642 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.302667    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-cd421c-openshift-default-dockercfg-thr5c\""
Oct 31 22:09:10.345001 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.344917    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlmxp\" (UniqueName: \"kubernetes.io/projected/be5c2058-2c58-4394-a4e9-c02e2aa18073-kube-api-access-qlmxp\") pod \"e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") " pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:09:10.345396 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.345370    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be5c2058-2c58-4394-a4e9-c02e2aa18073-istio-token\") pod \"e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") " pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:09:10.345711 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.345690    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-workload-certs\") pod \"e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") " pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:09:10.346037 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.346018    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be5c2058-2c58-4394-a4e9-c02e2aa18073-istio-podinfo\") pod \"e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") " pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:09:10.346327 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.346276    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-workload-socket\") pod \"e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") " pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:09:10.346550 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.346494    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-credential-socket\") pod \"e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") " pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:09:10.346719 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.346702    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-istio-data\") pod \"e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") " pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:09:10.347016 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.346998    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-istio-envoy\") pod \"e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") " pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:09:10.347392 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.347269    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be5c2058-2c58-4394-a4e9-c02e2aa18073-istiod-ca-cert\") pod \"e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") " pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:09:10.441833 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.441796    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:10.448712 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.448682    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-workload-socket\") pod \"e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") " pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:09:10.448973 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.448743    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-credential-socket\") pod \"e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") " pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:09:10.448973 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.448791    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-istio-data\") pod \"e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") " pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:09:10.448973 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.448851    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-istio-envoy\") pod \"e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") " pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:09:10.448973 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.448920    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be5c2058-2c58-4394-a4e9-c02e2aa18073-istiod-ca-cert\") pod \"e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") " pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:09:10.449265 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.449117    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlmxp\" (UniqueName: \"kubernetes.io/projected/be5c2058-2c58-4394-a4e9-c02e2aa18073-kube-api-access-qlmxp\") pod \"e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") " pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:09:10.449265 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.449168    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be5c2058-2c58-4394-a4e9-c02e2aa18073-istio-token\") pod \"e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") " pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:09:10.449265 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.449180    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-workload-socket\") pod \"e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") " pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:09:10.449265 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.449217    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-istio-data\") pod \"e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") " pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:09:10.449265 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.449238    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-workload-certs\") pod \"e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") " pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:09:10.449551 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.449333    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be5c2058-2c58-4394-a4e9-c02e2aa18073-istio-podinfo\") pod \"e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") " pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:09:10.449790 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.449761    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-workload-certs\") pod \"e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") " pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:09:10.449929 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.449906    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be5c2058-2c58-4394-a4e9-c02e2aa18073-istiod-ca-cert\") pod \"e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") " pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:09:10.450398 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.450332    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-credential-socket\") pod \"e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") " pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:09:10.452780 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.452753    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be5c2058-2c58-4394-a4e9-c02e2aa18073-istio-podinfo\") pod \"e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") " pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:09:10.453702 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.453671    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-istio-envoy\") pod \"e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") " pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:09:10.476150 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.476117    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlmxp\" (UniqueName: \"kubernetes.io/projected/be5c2058-2c58-4394-a4e9-c02e2aa18073-kube-api-access-qlmxp\") pod \"e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") " pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:09:10.476515 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.476489    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be5c2058-2c58-4394-a4e9-c02e2aa18073-istio-token\") pod \"e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") " pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:09:10.610780 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.610686    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:09:10.870188 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.870100    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:10.943551 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.943519    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"]
Oct 31 22:09:10.948898 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.948859    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:09:10.949209 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.949174    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:09:10.949340 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:10.949248    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:09:11.093319 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:11.093278    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg" event={"ID":"be5c2058-2c58-4394-a4e9-c02e2aa18073","Type":"ContainerStarted","Data":"d3d6f36ab726c7a88427c4ae544f4bf8603dcbc37e179a1674613a3f31549f91"}
Oct 31 22:09:11.093500 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:11.093335    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg" event={"ID":"be5c2058-2c58-4394-a4e9-c02e2aa18073","Type":"ContainerStarted","Data":"6c2917f5738b2825165aef200f833155eca0910e3868038e53a83149746a6f4f"}
Oct 31 22:09:11.142823 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:11.142653    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg" podStartSLOduration=1.142635298 podStartE2EDuration="1.142635298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:09:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:09:11.141230389 +0000 UTC m=+3191.057539820" watchObservedRunningTime="2026-10-31 22:09:11.142635298 +0000 UTC m=+3191.058944728"
Oct 31 22:09:11.611984 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:11.611954    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:09:12.608293 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:12.608260    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:12.618151 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:12.618123    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:09:13.108910 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:13.108877    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:09:13.109930 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:13.109910    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:09:13.495268 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:13.495178    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:13.632710 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:13.632669    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:13.798028 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:13.789574    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:14.708302 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:14.708263    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:15.777156 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:15.777111    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:15.896832 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:15.896790    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:16.749864 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:16.749814    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:17.097494 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:17.097460    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:17.499590 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:17.496198    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:17.873385 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:17.873345    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"]
Oct 31 22:09:17.878653 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:17.878629    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:09:17.900903 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:17.900871    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"]
Oct 31 22:09:17.916877 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:17.908783    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-c2d1b8-openshift-default-dockercfg-7999k\""
Oct 31 22:09:17.951989 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:17.951896    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:17.969695 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:17.969658    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-istio-envoy\") pod \"e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") " pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:09:17.969831 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:17.969719    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-workload-socket\") pod \"e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") " pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:09:17.969920 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:17.969902    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/876bda78-8ad9-445d-baa7-37c9ab1b4237-istiod-ca-cert\") pod \"e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") " pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:09:17.970060 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:17.970041    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-istio-data\") pod \"e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") " pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:09:17.970110 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:17.970079    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29ccp\" (UniqueName: \"kubernetes.io/projected/876bda78-8ad9-445d-baa7-37c9ab1b4237-kube-api-access-29ccp\") pod \"e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") " pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:09:17.970198 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:17.970182    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-workload-certs\") pod \"e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") " pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:09:17.970292 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:17.970274    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/876bda78-8ad9-445d-baa7-37c9ab1b4237-istio-token\") pod \"e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") " pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:09:17.970386 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:17.970372    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-credential-socket\") pod \"e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") " pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:09:17.970425 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:17.970403    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/876bda78-8ad9-445d-baa7-37c9ab1b4237-istio-podinfo\") pod \"e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") " pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:09:18.079083 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:18.079038    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/876bda78-8ad9-445d-baa7-37c9ab1b4237-istiod-ca-cert\") pod \"e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") " pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:09:18.079417 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:18.079381    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-istio-data\") pod \"e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") " pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:09:18.079609 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:18.079585    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29ccp\" (UniqueName: \"kubernetes.io/projected/876bda78-8ad9-445d-baa7-37c9ab1b4237-kube-api-access-29ccp\") pod \"e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") " pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:09:18.079793 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:18.079772    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-workload-certs\") pod \"e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") " pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:09:18.083960 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:18.079984    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/876bda78-8ad9-445d-baa7-37c9ab1b4237-istio-token\") pod \"e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") " pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:09:18.084223 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:18.084199    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-credential-socket\") pod \"e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") " pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:09:18.084373 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:18.084353    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/876bda78-8ad9-445d-baa7-37c9ab1b4237-istio-podinfo\") pod \"e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") " pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:09:18.084535 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:18.084518    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-istio-envoy\") pod \"e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") " pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:09:18.084663 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:18.084649    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-workload-socket\") pod \"e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") " pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:09:18.091964 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:18.085320    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-workload-socket\") pod \"e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") " pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:09:18.141339 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:18.092415    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-workload-certs\") pod \"e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") " pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:09:18.141339 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:18.093028    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/876bda78-8ad9-445d-baa7-37c9ab1b4237-istiod-ca-cert\") pod \"e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") " pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:09:18.141339 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:18.093255    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-istio-data\") pod \"e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") " pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:09:18.141339 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:18.093798    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-credential-socket\") pod \"e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") " pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:09:18.249754 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:18.249715    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/876bda78-8ad9-445d-baa7-37c9ab1b4237-istio-podinfo\") pod \"e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") " pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:09:18.253587 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:18.253554    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-istio-envoy\") pod \"e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") " pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:09:18.265669 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:18.265603    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29ccp\" (UniqueName: \"kubernetes.io/projected/876bda78-8ad9-445d-baa7-37c9ab1b4237-kube-api-access-29ccp\") pod \"e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") " pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:09:18.291989 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:18.289580    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:18.299395 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:18.299361    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/876bda78-8ad9-445d-baa7-37c9ab1b4237-istio-token\") pod \"e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") " pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:09:18.499155 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:18.499071    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:09:18.661030 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:18.658589    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:18.869481 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:18.869427    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"]
Oct 31 22:09:18.876006 ip-10-0-136-157 kubenswrapper[4460]: W1031 22:09:18.875968    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod876bda78_8ad9_445d_baa7_37c9ab1b4237.slice/crio-0630a84895a050dad98b56e43b66e0850f17dc448357f7eb3d62964e3f993a78 WatchSource:0}: Error finding container 0630a84895a050dad98b56e43b66e0850f17dc448357f7eb3d62964e3f993a78: Status 404 returned error can't find the container with id 0630a84895a050dad98b56e43b66e0850f17dc448357f7eb3d62964e3f993a78
Oct 31 22:09:18.878860 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:18.878823    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:09:18.878991 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:18.878896    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:09:18.878991 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:18.878927    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:09:19.313377 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:19.313328    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6" event={"ID":"876bda78-8ad9-445d-baa7-37c9ab1b4237","Type":"ContainerStarted","Data":"7acdecfc197a314529514a6f056144366fbe170877bb9087af1d0967d3bffa27"}
Oct 31 22:09:19.313869 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:19.313836    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6" event={"ID":"876bda78-8ad9-445d-baa7-37c9ab1b4237","Type":"ContainerStarted","Data":"0630a84895a050dad98b56e43b66e0850f17dc448357f7eb3d62964e3f993a78"}
Oct 31 22:09:19.371293 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:19.371236    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6" podStartSLOduration=2.371220476 podStartE2EDuration="2.371220476s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:09:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:09:19.367804918 +0000 UTC m=+3199.284114350" watchObservedRunningTime="2026-10-31 22:09:19.371220476 +0000 UTC m=+3199.287529905"
Oct 31 22:09:19.507402 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:19.507364    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:09:19.507807 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:19.507780    4460 patch_prober.go:29] interesting pod/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.215:15021/healthz/ready\": dial tcp 10.133.0.215:15021: connect: connection refused" start-of-body=
Oct 31 22:09:19.508142 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:19.508113    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6" podUID="876bda78-8ad9-445d-baa7-37c9ab1b4237" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.215:15021/healthz/ready\": dial tcp 10.133.0.215:15021: connect: connection refused"
Oct 31 22:09:20.384454 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:20.384412    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:20.508753 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:20.508140    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:09:21.352163 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:21.352131    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:09:21.353307 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:21.353285    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:09:21.490112 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:09:21.490060    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:09:21.667631 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:21.667526    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:21.782582 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:21.782544    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:22.322767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:22.322733    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:23.425622 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:23.425421    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:23.550382 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:23.550337    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:24.217421 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:24.217377    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:25.848014 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:25.847975    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:25.982182 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:25.982138    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:26.103037 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:26.102929    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:27.428245 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:27.428209    4460 generic.go:356] "Generic (PLEG): container finished" podID="2f79cdb1-ad35-4719-8f2b-591fa5965bc5" containerID="0246258b62f25177b9d52adc7f42966be057d5ba2aac2bae130835c6c169f299" exitCode=0
Oct 31 22:09:27.428823 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:27.428285    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs" event={"ID":"2f79cdb1-ad35-4719-8f2b-591fa5965bc5","Type":"ContainerDied","Data":"0246258b62f25177b9d52adc7f42966be057d5ba2aac2bae130835c6c169f299"}
Oct 31 22:09:27.428903 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:27.428856    4460 scope.go:122] "RemoveContainer" containerID="0246258b62f25177b9d52adc7f42966be057d5ba2aac2bae130835c6c169f299"
Oct 31 22:09:27.430820 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:27.430794    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:09:27.430900 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:27.430861    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:09:27.430900 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:27.430892    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:09:27.621958 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:27.621421    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:28.446834 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:28.446801    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs" event={"ID":"2f79cdb1-ad35-4719-8f2b-591fa5965bc5","Type":"ContainerStarted","Data":"d2c27a2e1df4c27f2a6f001cad7fe59f79c1ffcd718cef12018b0a61c4f2067d"}
Oct 31 22:09:28.585658 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:28.585619    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:09:28.585863 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:28.585839    4460 patch_prober.go:29] interesting pod/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.210:15021/healthz/ready\": dial tcp 10.133.0.210:15021: connect: connection refused" start-of-body=
Oct 31 22:09:28.585998 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:28.585910    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs" podUID="2f79cdb1-ad35-4719-8f2b-591fa5965bc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.210:15021/healthz/ready\": dial tcp 10.133.0.210:15021: connect: connection refused"
Oct 31 22:09:29.307055 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:29.307008    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:29.586315 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:29.586209    4460 patch_prober.go:29] interesting pod/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.210:15021/healthz/ready\": dial tcp 10.133.0.210:15021: connect: connection refused" start-of-body=
Oct 31 22:09:29.586315 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:29.586264    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs" podUID="2f79cdb1-ad35-4719-8f2b-591fa5965bc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.210:15021/healthz/ready\": dial tcp 10.133.0.210:15021: connect: connection refused"
Oct 31 22:09:30.585897 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:30.585859    4460 patch_prober.go:29] interesting pod/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.210:15021/healthz/ready\": dial tcp 10.133.0.210:15021: connect: connection refused" start-of-body=
Oct 31 22:09:30.586143 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:30.585932    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs" podUID="2f79cdb1-ad35-4719-8f2b-591fa5965bc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.210:15021/healthz/ready\": dial tcp 10.133.0.210:15021: connect: connection refused"
Oct 31 22:09:30.946233 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:30.946139    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:31.586386 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:31.586345    4460 patch_prober.go:29] interesting pod/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.210:15021/healthz/ready\": dial tcp 10.133.0.210:15021: connect: connection refused" start-of-body=
Oct 31 22:09:31.586616 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:31.586418    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs" podUID="2f79cdb1-ad35-4719-8f2b-591fa5965bc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.210:15021/healthz/ready\": dial tcp 10.133.0.210:15021: connect: connection refused"
Oct 31 22:09:32.586636 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:32.586580    4460 patch_prober.go:29] interesting pod/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.210:15021/healthz/ready\": dial tcp 10.133.0.210:15021: connect: connection refused" start-of-body=
Oct 31 22:09:32.587132 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:32.586658    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs" podUID="2f79cdb1-ad35-4719-8f2b-591fa5965bc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.210:15021/healthz/ready\": dial tcp 10.133.0.210:15021: connect: connection refused"
Oct 31 22:09:32.610211 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:32.610178    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:33.248391 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:33.248351    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 22:09:33.585826 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:33.585780    4460 patch_prober.go:29] interesting pod/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.210:15021/healthz/ready\": dial tcp 10.133.0.210:15021: connect: connection refused" start-of-body=
Oct 31 22:09:33.586058 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:33.585861    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs" podUID="2f79cdb1-ad35-4719-8f2b-591fa5965bc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.210:15021/healthz/ready\": dial tcp 10.133.0.210:15021: connect: connection refused"
Oct 31 22:09:34.009118 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:34.009012    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:09:34.042749 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:34.042705    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:34.652489 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:34.652414    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:34.771889 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:34.771831    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:34.808335 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:34.808306    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:09:34.821768 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:34.821735    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:09:34.884068 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:34.884034    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:35.278313 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:35.278277    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:36.944576 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:36.944496    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:37.152076 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:37.152028    4460 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podUID="9cd67bf5-8833-4a82-bc33-bed7bdfd4d36" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 22:09:37.152811 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:37.152342    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podUID="9cd67bf5-8833-4a82-bc33-bed7bdfd4d36" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": context deadline exceeded"
Oct 31 22:09:37.184072 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:37.184032    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-vnvpp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.204:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 22:09:37.184333 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:37.184301    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp" podUID="2412d51b-ffbb-496d-a28e-1a1b3868e98a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.204:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 22:09:37.252055 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:37.251933    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": context deadline exceeded"
Oct 31 22:09:37.430100 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:37.430052    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n" podUID="4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 22:09:37.452178 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:37.452118    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-b659t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.205:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 22:09:37.452567 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:37.452528    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t" podUID="db9cc38f-cdbc-4ae0-8645-cec895e76033" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.205:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 22:09:38.703365 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:38.703333    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:40.333907 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:40.333867    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:41.435311 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:41.435275    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:41.555456 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:41.555415    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:41.772246 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:41.772197    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:42.144456 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:42.144349    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:44.085433 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:44.085395    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:45.789709 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:45.789678    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:47.489866 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:47.489822    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:49.213485 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:49.213448    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:50.260100 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:50.260058    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:50.814293 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:50.814251    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:51.455176 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:09:51.455133    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:09:52.401923 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:52.401874    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:52.840375 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:52.840334    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-2dbae9-7dd5d8cd79-wjpd6"]
Oct 31 22:09:52.885784 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:52.885742    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-2dbae9-6ccd966867-7qlsd"]
Oct 31 22:09:52.988497 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:52.988458    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:53.123534 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:53.122525    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:54.102524 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:54.102481    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:55.845969 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:55.845894    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:55.987779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:55.987710    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:56.098711 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:56.098601    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:57.706563 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:57.706520    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:09:57.841666 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:57.841625    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-b-2dbae9-7dd5d8cd79-wjpd6" podUID="b79f9530-e471-49ab-acce-8391f7b8a91a" containerName="payload-processing" containerID="cri-o://e465aabd545c7154841b88aec6fb552a5ba3a6eef65a95bda6cab7ef84c9bd2e" gracePeriod=25
Oct 31 22:09:57.886776 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:57.886727    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-2dbae9-6ccd966867-7qlsd" podUID="eb66365f-35ce-4bb0-a756-9c2fba850b58" containerName="payload-pre-processing" containerID="cri-o://b09830e046b470f4de06274d04307fc9418f44f4c1493f491cff2460247835a9" gracePeriod=25
Oct 31 22:09:58.091353 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:58.091317    4460 generic.go:356] "Generic (PLEG): container finished" podID="eb66365f-35ce-4bb0-a756-9c2fba850b58" containerID="b09830e046b470f4de06274d04307fc9418f44f4c1493f491cff2460247835a9" exitCode=0
Oct 31 22:09:58.091579 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:58.091560    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-2dbae9-6ccd966867-7qlsd" event={"ID":"eb66365f-35ce-4bb0-a756-9c2fba850b58","Type":"ContainerDied","Data":"b09830e046b470f4de06274d04307fc9418f44f4c1493f491cff2460247835a9"}
Oct 31 22:09:58.093333 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:58.093308    4460 generic.go:356] "Generic (PLEG): container finished" podID="b79f9530-e471-49ab-acce-8391f7b8a91a" containerID="e465aabd545c7154841b88aec6fb552a5ba3a6eef65a95bda6cab7ef84c9bd2e" exitCode=0
Oct 31 22:09:58.093448 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:58.093339    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-2dbae9-7dd5d8cd79-wjpd6" event={"ID":"b79f9530-e471-49ab-acce-8391f7b8a91a","Type":"ContainerDied","Data":"e465aabd545c7154841b88aec6fb552a5ba3a6eef65a95bda6cab7ef84c9bd2e"}
Oct 31 22:09:58.225157 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:58.225129    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-2dbae9-7dd5d8cd79-wjpd6"
Oct 31 22:09:58.261677 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:58.261645    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-2dbae9-6ccd966867-7qlsd"
Oct 31 22:09:58.276752 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:58.276720    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b79f9530-e471-49ab-acce-8391f7b8a91a-kube-api-access-92jq4\" (UniqueName: \"kubernetes.io/projected/b79f9530-e471-49ab-acce-8391f7b8a91a-kube-api-access-92jq4\") pod \"b79f9530-e471-49ab-acce-8391f7b8a91a\" (UID: \"b79f9530-e471-49ab-acce-8391f7b8a91a\") "
Oct 31 22:09:58.276954 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:58.276804    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b79f9530-e471-49ab-acce-8391f7b8a91a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b79f9530-e471-49ab-acce-8391f7b8a91a-plugins-config-volume\") pod \"b79f9530-e471-49ab-acce-8391f7b8a91a\" (UID: \"b79f9530-e471-49ab-acce-8391f7b8a91a\") "
Oct 31 22:09:58.276954 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:58.276840    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb66365f-35ce-4bb0-a756-9c2fba850b58-kube-api-access-fn5ct\" (UniqueName: \"kubernetes.io/projected/eb66365f-35ce-4bb0-a756-9c2fba850b58-kube-api-access-fn5ct\") pod \"eb66365f-35ce-4bb0-a756-9c2fba850b58\" (UID: \"eb66365f-35ce-4bb0-a756-9c2fba850b58\") "
Oct 31 22:09:58.276954 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:58.276922    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eb66365f-35ce-4bb0-a756-9c2fba850b58-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb66365f-35ce-4bb0-a756-9c2fba850b58-plugins-config-volume\") pod \"eb66365f-35ce-4bb0-a756-9c2fba850b58\" (UID: \"eb66365f-35ce-4bb0-a756-9c2fba850b58\") "
Oct 31 22:09:58.277586 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:58.277553    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb66365f-35ce-4bb0-a756-9c2fba850b58-plugins-config-volume" pod "eb66365f-35ce-4bb0-a756-9c2fba850b58" (UID: "eb66365f-35ce-4bb0-a756-9c2fba850b58"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:09:58.277586 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:58.277575    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b79f9530-e471-49ab-acce-8391f7b8a91a-plugins-config-volume" pod "b79f9530-e471-49ab-acce-8391f7b8a91a" (UID: "b79f9530-e471-49ab-acce-8391f7b8a91a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:09:58.278303 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:58.278277    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb66365f-35ce-4bb0-a756-9c2fba850b58-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:09:58.278303 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:58.278304    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b79f9530-e471-49ab-acce-8391f7b8a91a-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:09:58.279430 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:58.279407    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b79f9530-e471-49ab-acce-8391f7b8a91a-kube-api-access-92jq4" pod "b79f9530-e471-49ab-acce-8391f7b8a91a" (UID: "b79f9530-e471-49ab-acce-8391f7b8a91a"). InnerVolumeSpecName "kube-api-access-92jq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:09:58.279642 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:58.279621    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb66365f-35ce-4bb0-a756-9c2fba850b58-kube-api-access-fn5ct" pod "eb66365f-35ce-4bb0-a756-9c2fba850b58" (UID: "eb66365f-35ce-4bb0-a756-9c2fba850b58"). InnerVolumeSpecName "kube-api-access-fn5ct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:09:58.379381 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:58.379348    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92jq4\" (UniqueName: \"kubernetes.io/projected/b79f9530-e471-49ab-acce-8391f7b8a91a-kube-api-access-92jq4\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:09:58.379381 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:58.379377    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fn5ct\" (UniqueName: \"kubernetes.io/projected/eb66365f-35ce-4bb0-a756-9c2fba850b58-kube-api-access-fn5ct\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:09:59.102152 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:59.102120    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-2dbae9-7dd5d8cd79-wjpd6"
Oct 31 22:09:59.102643 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:59.102165    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-2dbae9-7dd5d8cd79-wjpd6" event={"ID":"b79f9530-e471-49ab-acce-8391f7b8a91a","Type":"ContainerDied","Data":"c0ac3f36533f48df6a914e1b76463f6f422dbd2fed2786a44d8a593ee377b5e6"}
Oct 31 22:09:59.102643 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:59.102243    4460 scope.go:122] "RemoveContainer" containerID="e465aabd545c7154841b88aec6fb552a5ba3a6eef65a95bda6cab7ef84c9bd2e"
Oct 31 22:09:59.103762 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:59.103741    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-2dbae9-6ccd966867-7qlsd"
Oct 31 22:09:59.103879 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:59.103768    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-2dbae9-6ccd966867-7qlsd" event={"ID":"eb66365f-35ce-4bb0-a756-9c2fba850b58","Type":"ContainerDied","Data":"310ff336c351e29a52c458793bc1fb167fb5c3c6b25fa162193b486d59afaed4"}
Oct 31 22:09:59.113843 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:59.113827    4460 scope.go:122] "RemoveContainer" containerID="b09830e046b470f4de06274d04307fc9418f44f4c1493f491cff2460247835a9"
Oct 31 22:09:59.161436 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:59.161382    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-2dbae9-7dd5d8cd79-wjpd6"]
Oct 31 22:09:59.174860 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:59.174832    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-2dbae9-7dd5d8cd79-wjpd6"]
Oct 31 22:09:59.203678 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:59.203646    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-2dbae9-6ccd966867-7qlsd"]
Oct 31 22:09:59.208528 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:59.208502    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-2dbae9-6ccd966867-7qlsd"]
Oct 31 22:09:59.451882 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:09:59.451791    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:00.228532 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:00.228499    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b79f9530-e471-49ab-acce-8391f7b8a91a" path="/var/lib/kubelet/pods/b79f9530-e471-49ab-acce-8391f7b8a91a/volumes"
Oct 31 22:10:00.228997 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:00.228982    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eb66365f-35ce-4bb0-a756-9c2fba850b58" path="/var/lib/kubelet/pods/eb66365f-35ce-4bb0-a756-9c2fba850b58/volumes"
Oct 31 22:10:01.122183 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:01.122147    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:02.803961 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:02.803912    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:04.527532 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:04.527496    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:06.090309 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:06.090271    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:06.244421 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:06.243296    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:06.355476 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:06.355393    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:07.492462 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:07.492428    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:07.612622 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:07.612580    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:08.127635 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:08.127600    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:09.175652 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:09.175616    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:09.289403 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:09.289370    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:09.488231 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:09.488139    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:09.787491 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:09.787440    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:09.896026 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:09.895981    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:10.399814 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:10.399776    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:10.506502 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:10.506450    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:11.817401 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:11.817359    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:12.094457 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:12.094362    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:13.115557 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:13.115518    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:13.707337 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:13.707302    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:15.431952 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:15.431902    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:17.038455 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:17.038409    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:18.725240 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:18.725200    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:20.313971 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:20.313903    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:21.453370 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:10:21.453333    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:10:22.001770 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:22.001730    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:23.633137 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:23.633101    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:25.258925 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:25.258883    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:26.658375 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:26.658342    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:27.193345 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:27.193304    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:27.313184 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:27.313142    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:27.428779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:27.428738    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:27.816793 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:27.816762    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:30.500848 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:30.500812    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:31.012320 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:31.012282    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:31.893016 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:31.892484    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 22:10:32.718765 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:32.718726    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:34.555848 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:34.555815    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:34.877024 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:34.868509    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:34.994964 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:34.983901    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:35.157017 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:35.156019    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podUID="9cd67bf5-8833-4a82-bc33-bed7bdfd4d36" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": context deadline exceeded"
Oct 31 22:10:35.162327 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:35.162282    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747" podUID="d1078aa7-1947-45ce-b6bb-d6550ad61fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 22:10:35.205236 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:35.205189    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz" podUID="93a42128-ee74-4dcd-b158-27e33cfd44f9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 22:10:35.282963 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:35.267091    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 22:10:35.287781 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:35.287732    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:35.325383 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:35.325344    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n" podUID="4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": context deadline exceeded"
Oct 31 22:10:35.534961 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:35.532319    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt" podUID="9771611f-879a-44f6-9c1a-f8647dc2d841" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 22:10:35.608964 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:35.606479    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh" podUID="869ed893-352c-4f8f-a174-b42afdeb4de7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 22:10:35.748859 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:35.734041    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" podUID="af7e0e0c-cf52-4270-b528-35718339996c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Oct 31 22:10:35.975254 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:35.975136    4460 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 22:10:36.159974 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:36.158886    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podUID="9cd67bf5-8833-4a82-bc33-bed7bdfd4d36" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 22:10:36.159974 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:36.159486    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 22:10:36.164964 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:36.163018    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747" podUID="d1078aa7-1947-45ce-b6bb-d6550ad61fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 31 22:10:36.164964 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:36.163543    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 22:10:36.216972 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:36.211072    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz" podUID="93a42128-ee74-4dcd-b158-27e33cfd44f9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 22:10:36.218012 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:36.217986    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 22:10:36.330039 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:36.312479    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 22:10:36.630350 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:36.629035    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh" podUID="869ed893-352c-4f8f-a174-b42afdeb4de7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 22:10:36.736207 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:36.633183    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt" podUID="9771611f-879a-44f6-9c1a-f8647dc2d841" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 22:10:36.736207 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:36.681824    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n" podUID="4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": context deadline exceeded"
Oct 31 22:10:36.761962 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:36.741512    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" podUID="af7e0e0c-cf52-4270-b528-35718339996c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 22:10:36.761962 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:36.753600    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 22:10:36.761962 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:36.753663    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 22:10:36.761962 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:36.753702    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 22:10:36.761962 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:36.753736    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 22:10:36.761962 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:36.753775    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 22:10:36.761962 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:36.753809    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 22:10:36.877203 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:36.877163    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:37.079598 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:37.079566    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 22:10:37.182211 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:37.182168    4460 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podUID="9cd67bf5-8833-4a82-bc33-bed7bdfd4d36" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/health\": context deadline exceeded"
Oct 31 22:10:37.185774 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:37.185711    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747" podUID="d1078aa7-1947-45ce-b6bb-d6550ad61fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 31 22:10:37.186934 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:37.186901    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podUID="9cd67bf5-8833-4a82-bc33-bed7bdfd4d36" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 22:10:37.313966 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:37.313020    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 22:10:37.457508 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:37.457406    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-b659t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.205:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 22:10:37.479435 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:37.479368    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t" podUID="db9cc38f-cdbc-4ae0-8645-cec895e76033" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.205:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 22:10:37.641398 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:37.641353    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh" podUID="869ed893-352c-4f8f-a174-b42afdeb4de7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.77:8000/ready\": context deadline exceeded"
Oct 31 22:10:37.671168 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:37.671120    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt" podUID="9771611f-879a-44f6-9c1a-f8647dc2d841" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 22:10:37.717119 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:37.717023    4460 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-96tvd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.206:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 22:10:37.717382 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:37.717350    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd" podUID="2ce9ce6d-57d6-4085-94e5-2f7f600b81b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.206:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 22:10:37.774315 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:37.774260    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" podUID="af7e0e0c-cf52-4270-b528-35718339996c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 22:10:38.080095 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:38.080044    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n" podUID="4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 22:10:38.197967 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:38.196044    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747" podUID="d1078aa7-1947-45ce-b6bb-d6550ad61fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": context deadline exceeded"
Oct 31 22:10:38.197967 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:38.196974    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p" podUID="9cd67bf5-8833-4a82-bc33-bed7bdfd4d36" containerName="main" probeResult="failure" output="Get \"https://10.133.0.36:8000/ready\": context deadline exceeded"
Oct 31 22:10:38.311421 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:38.311384    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 22:10:38.341109 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:38.341020    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 22:10:38.356389 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:38.313280    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main" probeResult="failure" output="Get \"https://10.133.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 22:10:38.394060 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:38.394014    4460 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt" podUID="9771611f-879a-44f6-9c1a-f8647dc2d841" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/health\": context deadline exceeded"
Oct 31 22:10:38.413397 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:38.413367    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 22:10:38.503329 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:38.503283    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 22:10:38.614838 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:38.614753    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p"
Oct 31 22:10:38.615510 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:38.615487    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 22:10:38.701060 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:38.701008    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz" podUID="93a42128-ee74-4dcd-b158-27e33cfd44f9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 22:10:38.793441 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:38.793359    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:40.389063 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:40.388306    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:42.073300 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:42.073260    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:43.832716 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:43.832626    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:44.151179 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:44.151088    4460 patch_prober.go:29] interesting pod/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.214:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 22:10:44.151179 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:44.151165    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg" podUID="be5c2058-2c58-4394-a4e9-c02e2aa18073" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.214:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 22:10:45.239365 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:45.239319    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:45.334760 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:45.334715    4460 generic.go:356] "Generic (PLEG): container finished" podID="be5c2058-2c58-4394-a4e9-c02e2aa18073" containerID="d3d6f36ab726c7a88427c4ae544f4bf8603dcbc37e179a1674613a3f31549f91" exitCode=0
Oct 31 22:10:45.335995 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:45.334780    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg" event={"ID":"be5c2058-2c58-4394-a4e9-c02e2aa18073","Type":"ContainerDied","Data":"d3d6f36ab726c7a88427c4ae544f4bf8603dcbc37e179a1674613a3f31549f91"}
Oct 31 22:10:45.335995 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:45.335265    4460 scope.go:122] "RemoveContainer" containerID="d3d6f36ab726c7a88427c4ae544f4bf8603dcbc37e179a1674613a3f31549f91"
Oct 31 22:10:45.337300 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:45.337273    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:10:45.337382 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:45.337368    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:10:45.337425 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:45.337404    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:10:45.565324 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:45.565290    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:45.680678 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:45.678558    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:46.110805 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:46.110762    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:46.274197 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:46.270548    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:46.369425 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:46.369340    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-0877f5-6c8d64c49f-8t5wq"]
Oct 31 22:10:46.376037 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:46.376003    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0877f5-7b89c465c5-98ngt"]
Oct 31 22:10:46.396295 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:46.395284    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg" event={"ID":"be5c2058-2c58-4394-a4e9-c02e2aa18073","Type":"ContainerStarted","Data":"f6728ff416cceecec306dcafa426796dd99655792753ad82ab80d5fe451e6794"}
Oct 31 22:10:46.411597 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:46.398965    4460 generic.go:356] "Generic (PLEG): container finished" podID="876bda78-8ad9-445d-baa7-37c9ab1b4237" containerID="7acdecfc197a314529514a6f056144366fbe170877bb9087af1d0967d3bffa27" exitCode=0
Oct 31 22:10:46.411597 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:46.399134    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6" event={"ID":"876bda78-8ad9-445d-baa7-37c9ab1b4237","Type":"ContainerDied","Data":"7acdecfc197a314529514a6f056144366fbe170877bb9087af1d0967d3bffa27"}
Oct 31 22:10:46.411597 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:46.399653    4460 scope.go:122] "RemoveContainer" containerID="7acdecfc197a314529514a6f056144366fbe170877bb9087af1d0967d3bffa27"
Oct 31 22:10:46.414752 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:46.414706    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:10:46.415396 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:46.415322    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:10:46.415767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:46.415744    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:10:46.536416 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:46.536277    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:46.611025 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:46.610959    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:10:46.611283 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:46.611253    4460 patch_prober.go:29] interesting pod/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.214:15021/healthz/ready\": dial tcp 10.133.0.214:15021: connect: connection refused" start-of-body=
Oct 31 22:10:46.611393 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:46.611330    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg" podUID="be5c2058-2c58-4394-a4e9-c02e2aa18073" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.214:15021/healthz/ready\": dial tcp 10.133.0.214:15021: connect: connection refused"
Oct 31 22:10:46.664020 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:46.661294    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:47.338349 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:47.338318    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:47.450034 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:47.448119    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6" event={"ID":"876bda78-8ad9-445d-baa7-37c9ab1b4237","Type":"ContainerStarted","Data":"b43dcbff2f72f38b213b9d4a60464b0ac80c5ff97bad7c2b7bebc47ad4b08641"}
Oct 31 22:10:47.500120 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:47.500085    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:10:47.501701 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:47.501666    4460 patch_prober.go:29] interesting pod/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.215:15021/healthz/ready\": dial tcp 10.133.0.215:15021: connect: connection refused" start-of-body=
Oct 31 22:10:47.501861 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:47.501718    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6" podUID="876bda78-8ad9-445d-baa7-37c9ab1b4237" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.215:15021/healthz/ready\": dial tcp 10.133.0.215:15021: connect: connection refused"
Oct 31 22:10:47.611968 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:47.611856    4460 patch_prober.go:29] interesting pod/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.214:15021/healthz/ready\": dial tcp 10.133.0.214:15021: connect: connection refused" start-of-body=
Oct 31 22:10:47.611968 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:47.611904    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg" podUID="be5c2058-2c58-4394-a4e9-c02e2aa18073" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.214:15021/healthz/ready\": dial tcp 10.133.0.214:15021: connect: connection refused"
Oct 31 22:10:48.499609 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:48.499568    4460 patch_prober.go:29] interesting pod/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.215:15021/healthz/ready\": dial tcp 10.133.0.215:15021: connect: connection refused" start-of-body=
Oct 31 22:10:48.500062 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:48.499623    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6" podUID="876bda78-8ad9-445d-baa7-37c9ab1b4237" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.215:15021/healthz/ready\": dial tcp 10.133.0.215:15021: connect: connection refused"
Oct 31 22:10:48.611231 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:48.611194    4460 patch_prober.go:29] interesting pod/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.214:15021/healthz/ready\": dial tcp 10.133.0.214:15021: connect: connection refused" start-of-body=
Oct 31 22:10:48.611405 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:48.611263    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg" podUID="be5c2058-2c58-4394-a4e9-c02e2aa18073" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.214:15021/healthz/ready\": dial tcp 10.133.0.214:15021: connect: connection refused"
Oct 31 22:10:49.190625 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:49.190591    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:49.500081 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:49.499994    4460 patch_prober.go:29] interesting pod/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.215:15021/healthz/ready\": dial tcp 10.133.0.215:15021: connect: connection refused" start-of-body=
Oct 31 22:10:49.500081 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:49.500060    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6" podUID="876bda78-8ad9-445d-baa7-37c9ab1b4237" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.215:15021/healthz/ready\": dial tcp 10.133.0.215:15021: connect: connection refused"
Oct 31 22:10:49.612000 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:49.611963    4460 patch_prober.go:29] interesting pod/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.214:15021/healthz/ready\": dial tcp 10.133.0.214:15021: connect: connection refused" start-of-body=
Oct 31 22:10:49.612179 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:49.612020    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg" podUID="be5c2058-2c58-4394-a4e9-c02e2aa18073" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.214:15021/healthz/ready\": dial tcp 10.133.0.214:15021: connect: connection refused"
Oct 31 22:10:49.942964 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:49.942559    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:50.069635 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:50.069576    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:50.389085 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:50.389048    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:50.499974 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:50.499908    4460 patch_prober.go:29] interesting pod/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.215:15021/healthz/ready\": dial tcp 10.133.0.215:15021: connect: connection refused" start-of-body=
Oct 31 22:10:50.500181 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:50.500049    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6" podUID="876bda78-8ad9-445d-baa7-37c9ab1b4237" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.215:15021/healthz/ready\": dial tcp 10.133.0.215:15021: connect: connection refused"
Oct 31 22:10:50.611478 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:50.611444    4460 patch_prober.go:29] interesting pod/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.214:15021/healthz/ready\": dial tcp 10.133.0.214:15021: connect: connection refused" start-of-body=
Oct 31 22:10:50.611670 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:50.611496    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg" podUID="be5c2058-2c58-4394-a4e9-c02e2aa18073" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.214:15021/healthz/ready\": dial tcp 10.133.0.214:15021: connect: connection refused"
Oct 31 22:10:51.049605 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:51.049570    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:51.352158 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:51.352057    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:10:51.373056 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:51.373004    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0877f5-6c8d64c49f-8t5wq" podUID="249aadf1-c102-4ea3-b273-427c6df20f4f" containerName="payload-processing" containerID="cri-o://27c7b1d849fbd7b3316e5886979232a3f4df89a2fc6dc0d8d777614b15037da1" gracePeriod=25
Oct 31 22:10:51.377284 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:51.377253    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0877f5-7b89c465c5-98ngt" podUID="bb34199a-aed8-4d77-97bf-2e738b867c96" containerName="payload-pre-processing" containerID="cri-o://3344fc3644dbdf4625bd7578aa1a8ce69017ca24f9d58357b4a8248bbcf45da8" gracePeriod=25
Oct 31 22:10:51.496816 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:10:51.496778    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:10:51.500265 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:51.500236    4460 patch_prober.go:29] interesting pod/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.215:15021/healthz/ready\": dial tcp 10.133.0.215:15021: connect: connection refused" start-of-body=
Oct 31 22:10:51.500606 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:51.500283    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6" podUID="876bda78-8ad9-445d-baa7-37c9ab1b4237" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.215:15021/healthz/ready\": dial tcp 10.133.0.215:15021: connect: connection refused"
Oct 31 22:10:51.612535 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:51.612075    4460 patch_prober.go:29] interesting pod/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.214:15021/healthz/ready\": dial tcp 10.133.0.214:15021: connect: connection refused" start-of-body=
Oct 31 22:10:51.612535 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:51.612139    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg" podUID="be5c2058-2c58-4394-a4e9-c02e2aa18073" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.214:15021/healthz/ready\": dial tcp 10.133.0.214:15021: connect: connection refused"
Oct 31 22:10:51.902763 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:51.902669    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:52.065900 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:52.065860    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:52.096855 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:52.096820    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-c08360-8645d7ff59-s2tn2"]
Oct 31 22:10:52.097669 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:52.097653    4460 cpu_manager.go:414] "containerMap: removing container" podUID="b79f9530-e471-49ab-acce-8391f7b8a91a" containerName="payload-processing"
Oct 31 22:10:52.097723 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:52.097670    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b79f9530-e471-49ab-acce-8391f7b8a91a" containerName="payload-processing"
Oct 31 22:10:52.097723 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:52.097685    4460 cpu_manager.go:414] "containerMap: removing container" podUID="eb66365f-35ce-4bb0-a756-9c2fba850b58" containerName="payload-pre-processing"
Oct 31 22:10:52.097723 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:52.097691    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb66365f-35ce-4bb0-a756-9c2fba850b58" containerName="payload-pre-processing"
Oct 31 22:10:52.097859 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:52.097850    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="b79f9530-e471-49ab-acce-8391f7b8a91a" containerName="payload-processing"
Oct 31 22:10:52.097896 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:52.097861    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="eb66365f-35ce-4bb0-a756-9c2fba850b58" containerName="payload-pre-processing"
Oct 31 22:10:52.101050 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:52.101028    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c08360-8645d7ff59-s2tn2"
Oct 31 22:10:52.138265 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:52.138223    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-c08360\""
Oct 31 22:10:52.176071 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:52.158064    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-c08360-dockercfg-q9hh2\""
Oct 31 22:10:52.181082 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:52.178187    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-c08360-8645d7ff59-s2tn2"]
Oct 31 22:10:52.211603 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:52.211561    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lj4qz\" (UniqueName: \"kubernetes.io/projected/327e9388-0750-420d-ac21-2ee5fb4224d4-kube-api-access-lj4qz\") pod \"payload-pre-processing-e2e-inf-b-c08360-8645d7ff59-s2tn2\" (UID: \"327e9388-0750-420d-ac21-2ee5fb4224d4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c08360-8645d7ff59-s2tn2"
Oct 31 22:10:52.212002 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:52.211973    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/327e9388-0750-420d-ac21-2ee5fb4224d4-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-c08360-8645d7ff59-s2tn2\" (UID: \"327e9388-0750-420d-ac21-2ee5fb4224d4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c08360-8645d7ff59-s2tn2"
Oct 31 22:10:52.212746 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:52.212719    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c08360-58948b75c9-k66nk"]
Oct 31 22:10:52.386077 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:52.314753    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lj4qz\" (UniqueName: \"kubernetes.io/projected/327e9388-0750-420d-ac21-2ee5fb4224d4-kube-api-access-lj4qz\") pod \"payload-pre-processing-e2e-inf-b-c08360-8645d7ff59-s2tn2\" (UID: \"327e9388-0750-420d-ac21-2ee5fb4224d4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c08360-8645d7ff59-s2tn2"
Oct 31 22:10:52.386077 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:52.380315    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/327e9388-0750-420d-ac21-2ee5fb4224d4-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-c08360-8645d7ff59-s2tn2\" (UID: \"327e9388-0750-420d-ac21-2ee5fb4224d4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c08360-8645d7ff59-s2tn2"
Oct 31 22:10:52.386077 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:52.381548    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/327e9388-0750-420d-ac21-2ee5fb4224d4-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-c08360-8645d7ff59-s2tn2\" (UID: \"327e9388-0750-420d-ac21-2ee5fb4224d4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c08360-8645d7ff59-s2tn2"
Oct 31 22:10:52.497080 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:52.456220    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lj4qz\" (UniqueName: \"kubernetes.io/projected/327e9388-0750-420d-ac21-2ee5fb4224d4-kube-api-access-lj4qz\") pod \"payload-pre-processing-e2e-inf-b-c08360-8645d7ff59-s2tn2\" (UID: \"327e9388-0750-420d-ac21-2ee5fb4224d4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c08360-8645d7ff59-s2tn2"
Oct 31 22:10:52.500251 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:52.500194    4460 patch_prober.go:29] interesting pod/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.215:15021/healthz/ready\": dial tcp 10.133.0.215:15021: connect: connection refused" start-of-body=
Oct 31 22:10:52.500559 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:52.500528    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6" podUID="876bda78-8ad9-445d-baa7-37c9ab1b4237" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.215:15021/healthz/ready\": dial tcp 10.133.0.215:15021: connect: connection refused"
Oct 31 22:10:52.504248 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:52.504199    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c08360-58948b75c9-k66nk"
Oct 31 22:10:52.517795 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:52.517768    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c08360-58948b75c9-k66nk"]
Oct 31 22:10:52.590955 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:52.590741    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g8d2\" (UniqueName: \"kubernetes.io/projected/49303f39-9a79-4b1e-b8dc-12edad88fc96-kube-api-access-9g8d2\") pod \"payload-processing-e2e-inf-b-c08360-58948b75c9-k66nk\" (UID: \"49303f39-9a79-4b1e-b8dc-12edad88fc96\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c08360-58948b75c9-k66nk"
Oct 31 22:10:52.590955 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:52.590910    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49303f39-9a79-4b1e-b8dc-12edad88fc96-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-c08360-58948b75c9-k66nk\" (UID: \"49303f39-9a79-4b1e-b8dc-12edad88fc96\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c08360-58948b75c9-k66nk"
Oct 31 22:10:52.702963 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:52.700138    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9g8d2\" (UniqueName: \"kubernetes.io/projected/49303f39-9a79-4b1e-b8dc-12edad88fc96-kube-api-access-9g8d2\") pod \"payload-processing-e2e-inf-b-c08360-58948b75c9-k66nk\" (UID: \"49303f39-9a79-4b1e-b8dc-12edad88fc96\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c08360-58948b75c9-k66nk"
Oct 31 22:10:52.702963 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:52.700286    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49303f39-9a79-4b1e-b8dc-12edad88fc96-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-c08360-58948b75c9-k66nk\" (UID: \"49303f39-9a79-4b1e-b8dc-12edad88fc96\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c08360-58948b75c9-k66nk"
Oct 31 22:10:52.702963 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:52.701420    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49303f39-9a79-4b1e-b8dc-12edad88fc96-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-c08360-58948b75c9-k66nk\" (UID: \"49303f39-9a79-4b1e-b8dc-12edad88fc96\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c08360-58948b75c9-k66nk"
Oct 31 22:10:52.723963 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:52.719308    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c08360-8645d7ff59-s2tn2"
Oct 31 22:10:52.751223 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:52.741176    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g8d2\" (UniqueName: \"kubernetes.io/projected/49303f39-9a79-4b1e-b8dc-12edad88fc96-kube-api-access-9g8d2\") pod \"payload-processing-e2e-inf-b-c08360-58948b75c9-k66nk\" (UID: \"49303f39-9a79-4b1e-b8dc-12edad88fc96\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c08360-58948b75c9-k66nk"
Oct 31 22:10:52.851965 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:52.844533    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c08360-58948b75c9-k66nk"
Oct 31 22:10:52.887967 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:52.886957    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:53.075018 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:53.074959    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:10:53.076536 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:53.076503    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:10:53.079336 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:53.078920    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:10:53.262156 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:53.261814    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-c08360-8645d7ff59-s2tn2"]
Oct 31 22:10:53.262631 ip-10-0-136-157 kubenswrapper[4460]: W1031 22:10:53.262604    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod327e9388_0750_420d_ac21_2ee5fb4224d4.slice/crio-226890eb0520a541e04584bba263d92472af3fa6eba330e1ee072128104d20e1 WatchSource:0}: Error finding container 226890eb0520a541e04584bba263d92472af3fa6eba330e1ee072128104d20e1: Status 404 returned error can't find the container with id 226890eb0520a541e04584bba263d92472af3fa6eba330e1ee072128104d20e1
Oct 31 22:10:53.352114 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:53.352085    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c08360-58948b75c9-k66nk"]
Oct 31 22:10:53.355389 ip-10-0-136-157 kubenswrapper[4460]: W1031 22:10:53.355359    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49303f39_9a79_4b1e_b8dc_12edad88fc96.slice/crio-103c0a5aa917ad41a28d1eb3f02ea5a028e07db6c81396f9f7b9f88e86d5492b WatchSource:0}: Error finding container 103c0a5aa917ad41a28d1eb3f02ea5a028e07db6c81396f9f7b9f88e86d5492b: Status 404 returned error can't find the container with id 103c0a5aa917ad41a28d1eb3f02ea5a028e07db6c81396f9f7b9f88e86d5492b
Oct 31 22:10:53.463580 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:53.463532    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:53.565750 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:53.563236    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:10:53.571069 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:53.568767    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:10:53.571621 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:53.571588    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c08360-8645d7ff59-s2tn2" event={"ID":"327e9388-0750-420d-ac21-2ee5fb4224d4","Type":"ContainerStarted","Data":"c46c876c32c20c0de3a882e2489b5b8e4bb84b53ca4ac0d778cf288862564435"}
Oct 31 22:10:53.571731 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:53.571648    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c08360-8645d7ff59-s2tn2" event={"ID":"327e9388-0750-420d-ac21-2ee5fb4224d4","Type":"ContainerStarted","Data":"226890eb0520a541e04584bba263d92472af3fa6eba330e1ee072128104d20e1"}
Oct 31 22:10:53.573982 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:53.572503    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c08360-8645d7ff59-s2tn2"
Oct 31 22:10:53.575525 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:53.575503    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c08360-58948b75c9-k66nk" event={"ID":"49303f39-9a79-4b1e-b8dc-12edad88fc96","Type":"ContainerStarted","Data":"04a2df8b073f6013d67c391efee51e660d3533928ea88baf9d832ea7031d163b"}
Oct 31 22:10:53.575592 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:53.575541    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c08360-58948b75c9-k66nk" event={"ID":"49303f39-9a79-4b1e-b8dc-12edad88fc96","Type":"ContainerStarted","Data":"103c0a5aa917ad41a28d1eb3f02ea5a028e07db6c81396f9f7b9f88e86d5492b"}
Oct 31 22:10:53.575592 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:53.575558    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-c08360-58948b75c9-k66nk"
Oct 31 22:10:53.661557 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:53.661496    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c08360-8645d7ff59-s2tn2" podStartSLOduration=1.661480895 podStartE2EDuration="1.661480895s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:10:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:10:53.655984063 +0000 UTC m=+3293.572293494" watchObservedRunningTime="2026-10-31 22:10:53.661480895 +0000 UTC m=+3293.577790325"
Oct 31 22:10:53.769553 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:53.769484    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-c08360-58948b75c9-k66nk" podStartSLOduration=1.769464067 podStartE2EDuration="1.769464067s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:10:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:10:53.763797855 +0000 UTC m=+3293.680107287" watchObservedRunningTime="2026-10-31 22:10:53.769464067 +0000 UTC m=+3293.685773498"
Oct 31 22:10:54.802925 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:54.802889    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:55.476639 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:55.476605    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:56.349670 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:56.349629    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0877f5-6c8d64c49f-8t5wq" podUID="249aadf1-c102-4ea3-b273-427c6df20f4f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:10:56.349670 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:56.349651    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0877f5-7b89c465c5-98ngt" podUID="bb34199a-aed8-4d77-97bf-2e738b867c96" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:10:56.423006 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:56.422970    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:58.128909 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:58.128872    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:10:59.825527 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:10:59.825484    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:00.871244 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:00.871214    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ssl9_26069598-5dc4-4c96-b55f-dd2f09884419/ovn-acl-logging/0.log"
Oct 31 22:11:00.879097 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:00.879073    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ssl9_26069598-5dc4-4c96-b55f-dd2f09884419/ovn-acl-logging/0.log"
Oct 31 22:11:01.502662 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:01.502623    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:03.007107 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:03.007068    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:03.131330 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:03.130895    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:03.401080 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:03.400924    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:04.585187 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:04.585151    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-c08360-58948b75c9-k66nk"
Oct 31 22:11:04.844152 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:04.844063    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:05.307113 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:05.307079    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:05.594611 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:05.594525    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c08360-8645d7ff59-s2tn2"
Oct 31 22:11:05.896373 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:05.896280    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:06.023822 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:06.023717    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:06.349132 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:06.349089    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0877f5-6c8d64c49f-8t5wq" podUID="249aadf1-c102-4ea3-b273-427c6df20f4f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:11:06.349132 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:06.349111    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0877f5-7b89c465c5-98ngt" podUID="bb34199a-aed8-4d77-97bf-2e738b867c96" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:11:07.118714 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:07.118669    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:07.416029 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:07.415859    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:07.531963 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:07.530960    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:07.702255 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:07.702120    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:08.013023 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:08.009386    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:08.568743 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:08.568706    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:08.920582 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:08.920491    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:10.752324 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:10.752257    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:11.688252 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:11.688220    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0877f5-6c8d64c49f-8t5wq"
Oct 31 22:11:11.707469 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:11.707442    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0877f5-7b89c465c5-98ngt"
Oct 31 22:11:11.763110 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:11.763066    4460 generic.go:356] "Generic (PLEG): container finished" podID="249aadf1-c102-4ea3-b273-427c6df20f4f" containerID="27c7b1d849fbd7b3316e5886979232a3f4df89a2fc6dc0d8d777614b15037da1" exitCode=0
Oct 31 22:11:11.763640 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:11.763120    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0877f5-6c8d64c49f-8t5wq"
Oct 31 22:11:11.763640 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:11.763152    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0877f5-6c8d64c49f-8t5wq" event={"ID":"249aadf1-c102-4ea3-b273-427c6df20f4f","Type":"ContainerDied","Data":"27c7b1d849fbd7b3316e5886979232a3f4df89a2fc6dc0d8d777614b15037da1"}
Oct 31 22:11:11.763640 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:11.763249    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-0877f5-6c8d64c49f-8t5wq" event={"ID":"249aadf1-c102-4ea3-b273-427c6df20f4f","Type":"ContainerDied","Data":"6a8d6569f6c119d40fe5b7b3ad6dda53940be27ea21579feb995b7651c349da7"}
Oct 31 22:11:11.763640 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:11.763280    4460 scope.go:122] "RemoveContainer" containerID="27c7b1d849fbd7b3316e5886979232a3f4df89a2fc6dc0d8d777614b15037da1"
Oct 31 22:11:11.765018 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:11.764993    4460 generic.go:356] "Generic (PLEG): container finished" podID="bb34199a-aed8-4d77-97bf-2e738b867c96" containerID="3344fc3644dbdf4625bd7578aa1a8ce69017ca24f9d58357b4a8248bbcf45da8" exitCode=0
Oct 31 22:11:11.765158 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:11.765072    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0877f5-7b89c465c5-98ngt" event={"ID":"bb34199a-aed8-4d77-97bf-2e738b867c96","Type":"ContainerDied","Data":"3344fc3644dbdf4625bd7578aa1a8ce69017ca24f9d58357b4a8248bbcf45da8"}
Oct 31 22:11:11.765158 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:11.765122    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0877f5-7b89c465c5-98ngt" event={"ID":"bb34199a-aed8-4d77-97bf-2e738b867c96","Type":"ContainerDied","Data":"5d4bc7ccfb8d0c8033d9668cb435c45e7fdb32185d14241d99017b7c7dcc3a6a"}
Oct 31 22:11:11.765307 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:11.765288    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0877f5-7b89c465c5-98ngt"
Oct 31 22:11:11.774549 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:11.774520    4460 scope.go:122] "RemoveContainer" containerID="27c7b1d849fbd7b3316e5886979232a3f4df89a2fc6dc0d8d777614b15037da1"
Oct 31 22:11:11.775118 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:11:11.775090    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27c7b1d849fbd7b3316e5886979232a3f4df89a2fc6dc0d8d777614b15037da1\": container with ID starting with 27c7b1d849fbd7b3316e5886979232a3f4df89a2fc6dc0d8d777614b15037da1 not found: ID does not exist" containerID="27c7b1d849fbd7b3316e5886979232a3f4df89a2fc6dc0d8d777614b15037da1"
Oct 31 22:11:11.775267 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:11.775128    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27c7b1d849fbd7b3316e5886979232a3f4df89a2fc6dc0d8d777614b15037da1"} err="failed to get container status \"27c7b1d849fbd7b3316e5886979232a3f4df89a2fc6dc0d8d777614b15037da1\": rpc error: code = NotFound desc = could not find container \"27c7b1d849fbd7b3316e5886979232a3f4df89a2fc6dc0d8d777614b15037da1\": container with ID starting with 27c7b1d849fbd7b3316e5886979232a3f4df89a2fc6dc0d8d777614b15037da1 not found: ID does not exist"
Oct 31 22:11:11.775267 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:11.775156    4460 scope.go:122] "RemoveContainer" containerID="3344fc3644dbdf4625bd7578aa1a8ce69017ca24f9d58357b4a8248bbcf45da8"
Oct 31 22:11:11.808426 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:11.808396    4460 scope.go:122] "RemoveContainer" containerID="3344fc3644dbdf4625bd7578aa1a8ce69017ca24f9d58357b4a8248bbcf45da8"
Oct 31 22:11:11.808856 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:11:11.808826    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3344fc3644dbdf4625bd7578aa1a8ce69017ca24f9d58357b4a8248bbcf45da8\": container with ID starting with 3344fc3644dbdf4625bd7578aa1a8ce69017ca24f9d58357b4a8248bbcf45da8 not found: ID does not exist" containerID="3344fc3644dbdf4625bd7578aa1a8ce69017ca24f9d58357b4a8248bbcf45da8"
Oct 31 22:11:11.808934 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:11.808869    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3344fc3644dbdf4625bd7578aa1a8ce69017ca24f9d58357b4a8248bbcf45da8"} err="failed to get container status \"3344fc3644dbdf4625bd7578aa1a8ce69017ca24f9d58357b4a8248bbcf45da8\": rpc error: code = NotFound desc = could not find container \"3344fc3644dbdf4625bd7578aa1a8ce69017ca24f9d58357b4a8248bbcf45da8\": container with ID starting with 3344fc3644dbdf4625bd7578aa1a8ce69017ca24f9d58357b4a8248bbcf45da8 not found: ID does not exist"
Oct 31 22:11:11.815326 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:11.815293    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/249aadf1-c102-4ea3-b273-427c6df20f4f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/249aadf1-c102-4ea3-b273-427c6df20f4f-plugins-config-volume\") pod \"249aadf1-c102-4ea3-b273-427c6df20f4f\" (UID: \"249aadf1-c102-4ea3-b273-427c6df20f4f\") "
Oct 31 22:11:11.815492 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:11.815366    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/249aadf1-c102-4ea3-b273-427c6df20f4f-kube-api-access-hp4pn\" (UniqueName: \"kubernetes.io/projected/249aadf1-c102-4ea3-b273-427c6df20f4f-kube-api-access-hp4pn\") pod \"249aadf1-c102-4ea3-b273-427c6df20f4f\" (UID: \"249aadf1-c102-4ea3-b273-427c6df20f4f\") "
Oct 31 22:11:11.815591 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:11.815577    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bb34199a-aed8-4d77-97bf-2e738b867c96-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb34199a-aed8-4d77-97bf-2e738b867c96-plugins-config-volume\") pod \"bb34199a-aed8-4d77-97bf-2e738b867c96\" (UID: \"bb34199a-aed8-4d77-97bf-2e738b867c96\") "
Oct 31 22:11:11.815683 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:11.815668    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bb34199a-aed8-4d77-97bf-2e738b867c96-kube-api-access-575n8\" (UniqueName: \"kubernetes.io/projected/bb34199a-aed8-4d77-97bf-2e738b867c96-kube-api-access-575n8\") pod \"bb34199a-aed8-4d77-97bf-2e738b867c96\" (UID: \"bb34199a-aed8-4d77-97bf-2e738b867c96\") "
Oct 31 22:11:11.815756 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:11.815730    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/249aadf1-c102-4ea3-b273-427c6df20f4f-plugins-config-volume" pod "249aadf1-c102-4ea3-b273-427c6df20f4f" (UID: "249aadf1-c102-4ea3-b273-427c6df20f4f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:11:11.816030 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:11.816004    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb34199a-aed8-4d77-97bf-2e738b867c96-plugins-config-volume" pod "bb34199a-aed8-4d77-97bf-2e738b867c96" (UID: "bb34199a-aed8-4d77-97bf-2e738b867c96"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:11:11.816369 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:11.816350    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb34199a-aed8-4d77-97bf-2e738b867c96-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:11:11.816369 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:11.816370    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/249aadf1-c102-4ea3-b273-427c6df20f4f-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:11:11.817977 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:11.817921    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/249aadf1-c102-4ea3-b273-427c6df20f4f-kube-api-access-hp4pn" pod "249aadf1-c102-4ea3-b273-427c6df20f4f" (UID: "249aadf1-c102-4ea3-b273-427c6df20f4f"). InnerVolumeSpecName "kube-api-access-hp4pn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:11:11.818276 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:11.818254    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb34199a-aed8-4d77-97bf-2e738b867c96-kube-api-access-575n8" pod "bb34199a-aed8-4d77-97bf-2e738b867c96" (UID: "bb34199a-aed8-4d77-97bf-2e738b867c96"). InnerVolumeSpecName "kube-api-access-575n8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:11:11.918459 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:11.918378    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-575n8\" (UniqueName: \"kubernetes.io/projected/bb34199a-aed8-4d77-97bf-2e738b867c96-kube-api-access-575n8\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:11:11.918459 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:11.918410    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hp4pn\" (UniqueName: \"kubernetes.io/projected/249aadf1-c102-4ea3-b273-427c6df20f4f-kube-api-access-hp4pn\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:11:12.102748 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:12.102712    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:12.126792 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:12.126758    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-0877f5-6c8d64c49f-8t5wq"]
Oct 31 22:11:12.132861 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:12.132833    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-0877f5-6c8d64c49f-8t5wq"]
Oct 31 22:11:12.170033 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:12.169970    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0877f5-7b89c465c5-98ngt"]
Oct 31 22:11:12.175011 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:12.174986    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-0877f5-7b89c465c5-98ngt"]
Oct 31 22:11:12.227850 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:12.227818    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="249aadf1-c102-4ea3-b273-427c6df20f4f" path="/var/lib/kubelet/pods/249aadf1-c102-4ea3-b273-427c6df20f4f/volumes"
Oct 31 22:11:12.228311 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:12.228295    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bb34199a-aed8-4d77-97bf-2e738b867c96" path="/var/lib/kubelet/pods/bb34199a-aed8-4d77-97bf-2e738b867c96/volumes"
Oct 31 22:11:12.705795 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:12.705754    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:12.817124 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:12.817085    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:12.936963 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:12.934382    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:13.332254 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:13.332222    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:14.781227 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:14.781188    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:15.995018 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:15.994976    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:16.120256 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:16.120218    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:16.436179 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:16.436138    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:16.542870 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:16.542838    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:16.942006 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:16.941966    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:17.383185 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:17.383150    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:18.186672 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:18.186633    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:18.441080 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:18.440974    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:20.090744 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:20.090701    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:20.206315 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:20.206281    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:20.328661 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:20.328608    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:20.446078 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:20.445982    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:20.756043 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:20.754601    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:20.868859 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:20.868827    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:21.455044 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:11:21.455012    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:11:22.173605 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:22.173571    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:22.618999 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:22.618913    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:22.730170 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:22.730136    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-1b0cce-7996cb486f-vpwrv"]
Oct 31 22:11:22.730979 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:22.730958    4460 cpu_manager.go:414] "containerMap: removing container" podUID="bb34199a-aed8-4d77-97bf-2e738b867c96" containerName="payload-pre-processing"
Oct 31 22:11:22.731114 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:22.730982    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bb34199a-aed8-4d77-97bf-2e738b867c96" containerName="payload-pre-processing"
Oct 31 22:11:22.731114 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:22.731028    4460 cpu_manager.go:414] "containerMap: removing container" podUID="249aadf1-c102-4ea3-b273-427c6df20f4f" containerName="payload-processing"
Oct 31 22:11:22.731114 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:22.731034    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="249aadf1-c102-4ea3-b273-427c6df20f4f" containerName="payload-processing"
Oct 31 22:11:22.731319 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:22.731292    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="bb34199a-aed8-4d77-97bf-2e738b867c96" containerName="payload-pre-processing"
Oct 31 22:11:22.731379 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:22.731322    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="249aadf1-c102-4ea3-b273-427c6df20f4f" containerName="payload-processing"
Oct 31 22:11:22.736872 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:22.736853    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-1b0cce-7996cb486f-vpwrv"
Oct 31 22:11:22.751045 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:22.751009    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:22.753145 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:22.753117    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-1b0cce-7996cb486f-vpwrv"]
Oct 31 22:11:22.761685 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:22.761659    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-1b0cce\""
Oct 31 22:11:22.799167 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:22.799131    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-1b0cce-7f88b8d6dc-v4qx7"]
Oct 31 22:11:22.803986 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:22.803926    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1b0cce-7f88b8d6dc-v4qx7"
Oct 31 22:11:22.822446 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:22.822417    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-1b0cce-7f88b8d6dc-v4qx7"]
Oct 31 22:11:22.841419 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:22.841384    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-1b0cce-dockercfg-ztmw6\""
Oct 31 22:11:22.972073 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:22.968781    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nx5f\" (UniqueName: \"kubernetes.io/projected/47f0f7a9-1ed4-4046-898f-29220ce6f46b-kube-api-access-2nx5f\") pod \"payload-processing-e2e-inf-a-1b0cce-7996cb486f-vpwrv\" (UID: \"47f0f7a9-1ed4-4046-898f-29220ce6f46b\") " pod="openshift-ingress/payload-processing-e2e-inf-a-1b0cce-7996cb486f-vpwrv"
Oct 31 22:11:22.972073 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:22.968921    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47f0f7a9-1ed4-4046-898f-29220ce6f46b-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-1b0cce-7996cb486f-vpwrv\" (UID: \"47f0f7a9-1ed4-4046-898f-29220ce6f46b\") " pod="openshift-ingress/payload-processing-e2e-inf-a-1b0cce-7996cb486f-vpwrv"
Oct 31 22:11:22.972073 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:22.969200    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa2bcc59-339d-499c-84f0-b2a6eb365ec0-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-1b0cce-7f88b8d6dc-v4qx7\" (UID: \"aa2bcc59-339d-499c-84f0-b2a6eb365ec0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1b0cce-7f88b8d6dc-v4qx7"
Oct 31 22:11:22.972073 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:22.969228    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ksnp\" (UniqueName: \"kubernetes.io/projected/aa2bcc59-339d-499c-84f0-b2a6eb365ec0-kube-api-access-9ksnp\") pod \"payload-pre-processing-e2e-inf-a-1b0cce-7f88b8d6dc-v4qx7\" (UID: \"aa2bcc59-339d-499c-84f0-b2a6eb365ec0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1b0cce-7f88b8d6dc-v4qx7"
Oct 31 22:11:23.070784 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:23.070736    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nx5f\" (UniqueName: \"kubernetes.io/projected/47f0f7a9-1ed4-4046-898f-29220ce6f46b-kube-api-access-2nx5f\") pod \"payload-processing-e2e-inf-a-1b0cce-7996cb486f-vpwrv\" (UID: \"47f0f7a9-1ed4-4046-898f-29220ce6f46b\") " pod="openshift-ingress/payload-processing-e2e-inf-a-1b0cce-7996cb486f-vpwrv"
Oct 31 22:11:23.070985 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:23.070806    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47f0f7a9-1ed4-4046-898f-29220ce6f46b-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-1b0cce-7996cb486f-vpwrv\" (UID: \"47f0f7a9-1ed4-4046-898f-29220ce6f46b\") " pod="openshift-ingress/payload-processing-e2e-inf-a-1b0cce-7996cb486f-vpwrv"
Oct 31 22:11:23.071062 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:23.071023    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa2bcc59-339d-499c-84f0-b2a6eb365ec0-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-1b0cce-7f88b8d6dc-v4qx7\" (UID: \"aa2bcc59-339d-499c-84f0-b2a6eb365ec0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1b0cce-7f88b8d6dc-v4qx7"
Oct 31 22:11:23.071062 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:23.071050    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ksnp\" (UniqueName: \"kubernetes.io/projected/aa2bcc59-339d-499c-84f0-b2a6eb365ec0-kube-api-access-9ksnp\") pod \"payload-pre-processing-e2e-inf-a-1b0cce-7f88b8d6dc-v4qx7\" (UID: \"aa2bcc59-339d-499c-84f0-b2a6eb365ec0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1b0cce-7f88b8d6dc-v4qx7"
Oct 31 22:11:23.072474 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:23.072445    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47f0f7a9-1ed4-4046-898f-29220ce6f46b-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-1b0cce-7996cb486f-vpwrv\" (UID: \"47f0f7a9-1ed4-4046-898f-29220ce6f46b\") " pod="openshift-ingress/payload-processing-e2e-inf-a-1b0cce-7996cb486f-vpwrv"
Oct 31 22:11:23.072669 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:23.072651    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa2bcc59-339d-499c-84f0-b2a6eb365ec0-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-1b0cce-7f88b8d6dc-v4qx7\" (UID: \"aa2bcc59-339d-499c-84f0-b2a6eb365ec0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1b0cce-7f88b8d6dc-v4qx7"
Oct 31 22:11:23.096647 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:23.096622    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nx5f\" (UniqueName: \"kubernetes.io/projected/47f0f7a9-1ed4-4046-898f-29220ce6f46b-kube-api-access-2nx5f\") pod \"payload-processing-e2e-inf-a-1b0cce-7996cb486f-vpwrv\" (UID: \"47f0f7a9-1ed4-4046-898f-29220ce6f46b\") " pod="openshift-ingress/payload-processing-e2e-inf-a-1b0cce-7996cb486f-vpwrv"
Oct 31 22:11:23.096802 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:23.096779    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ksnp\" (UniqueName: \"kubernetes.io/projected/aa2bcc59-339d-499c-84f0-b2a6eb365ec0-kube-api-access-9ksnp\") pod \"payload-pre-processing-e2e-inf-a-1b0cce-7f88b8d6dc-v4qx7\" (UID: \"aa2bcc59-339d-499c-84f0-b2a6eb365ec0\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1b0cce-7f88b8d6dc-v4qx7"
Oct 31 22:11:23.124361 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:23.124329    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1b0cce-7f88b8d6dc-v4qx7"
Oct 31 22:11:23.353399 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:23.353370    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-1b0cce-7996cb486f-vpwrv"
Oct 31 22:11:23.436774 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:23.435719    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-1b0cce-7f88b8d6dc-v4qx7"]
Oct 31 22:11:23.439927 ip-10-0-136-157 kubenswrapper[4460]: W1031 22:11:23.439896    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa2bcc59_339d_499c_84f0_b2a6eb365ec0.slice/crio-efbb05c78e787d214483895f299e471cce44961de2900250ebd5d5b1ddf54e21 WatchSource:0}: Error finding container efbb05c78e787d214483895f299e471cce44961de2900250ebd5d5b1ddf54e21: Status 404 returned error can't find the container with id efbb05c78e787d214483895f299e471cce44961de2900250ebd5d5b1ddf54e21
Oct 31 22:11:23.660332 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:23.660300    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-1b0cce-7996cb486f-vpwrv"]
Oct 31 22:11:23.660674 ip-10-0-136-157 kubenswrapper[4460]: W1031 22:11:23.660428    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47f0f7a9_1ed4_4046_898f_29220ce6f46b.slice/crio-9718e0e63f3e0d116c66bf30f8580dab0d7e401e677a8c5bd66555f24b1bf85c WatchSource:0}: Error finding container 9718e0e63f3e0d116c66bf30f8580dab0d7e401e677a8c5bd66555f24b1bf85c: Status 404 returned error can't find the container with id 9718e0e63f3e0d116c66bf30f8580dab0d7e401e677a8c5bd66555f24b1bf85c
Oct 31 22:11:24.007880 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:24.007799    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1b0cce-7f88b8d6dc-v4qx7" event={"ID":"aa2bcc59-339d-499c-84f0-b2a6eb365ec0","Type":"ContainerStarted","Data":"c511e565f8e548d6d04167b629cf48e38975dadbe9ac5cab4c4c058584fee7d5"}
Oct 31 22:11:24.007880 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:24.007862    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1b0cce-7f88b8d6dc-v4qx7" event={"ID":"aa2bcc59-339d-499c-84f0-b2a6eb365ec0","Type":"ContainerStarted","Data":"efbb05c78e787d214483895f299e471cce44961de2900250ebd5d5b1ddf54e21"}
Oct 31 22:11:24.008125 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:24.007895    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1b0cce-7f88b8d6dc-v4qx7"
Oct 31 22:11:24.009731 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:24.009694    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-1b0cce-7996cb486f-vpwrv" event={"ID":"47f0f7a9-1ed4-4046-898f-29220ce6f46b","Type":"ContainerStarted","Data":"f4004f5c1898b425d9781d619d9cbc1fb2b500e596cc18fbbedc4ba10df14f7d"}
Oct 31 22:11:24.009851 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:24.009763    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-1b0cce-7996cb486f-vpwrv"
Oct 31 22:11:24.009851 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:24.009788    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-1b0cce-7996cb486f-vpwrv" event={"ID":"47f0f7a9-1ed4-4046-898f-29220ce6f46b","Type":"ContainerStarted","Data":"9718e0e63f3e0d116c66bf30f8580dab0d7e401e677a8c5bd66555f24b1bf85c"}
Oct 31 22:11:24.053774 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:24.053727    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1b0cce-7f88b8d6dc-v4qx7" podStartSLOduration=2.053712882 podStartE2EDuration="2.053712882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:11:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:11:24.051151179 +0000 UTC m=+3323.967460610" watchObservedRunningTime="2026-10-31 22:11:24.053712882 +0000 UTC m=+3323.970022312"
Oct 31 22:11:24.096052 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:24.095997    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-1b0cce-7996cb486f-vpwrv" podStartSLOduration=2.095983268 podStartE2EDuration="2.095983268s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:11:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:11:24.094107503 +0000 UTC m=+3324.010416942" watchObservedRunningTime="2026-10-31 22:11:24.095983268 +0000 UTC m=+3324.012292698"
Oct 31 22:11:24.099332 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:24.099300    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:25.747684 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:25.747646    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:27.332467 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:27.332436    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:28.930905 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:28.930855    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:30.513577 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:30.513544    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:31.600088 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:31.600051    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:31.735088 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:31.733421    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:32.370794 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:32.354368    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:32.483642 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:32.483603    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:32.590268 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:32.590237    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:33.969792 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:33.969751    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:35.021102 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:35.021070    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-1b0cce-7996cb486f-vpwrv"
Oct 31 22:11:35.021500 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:35.021438    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1b0cce-7f88b8d6dc-v4qx7"
Oct 31 22:11:35.695759 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:35.695717    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:37.284169 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:37.284118    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:38.645102 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:38.645062    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:39.256345 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:39.256287    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:39.370803 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:39.370767    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:39.824289 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:39.824249    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:41.239991 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:41.239927    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:42.928558 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:42.928520    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:44.545640 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:44.545601    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:46.336729 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:46.336690    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:46.638057 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:46.637970    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:46.747919 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:46.747876    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:46.948453 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:46.948366    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"]
Oct 31 22:11:46.948958 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:46.948907    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x" podUID="60161a74-1f84-4f09-9a66-b0c1d1fdce14" containerName="istio-proxy" containerID="cri-o://03b7e02788741058f0b69f5b5a10c4ca570ec8f5a6bd65e4df21e02fe9b83f59" gracePeriod=30
Oct 31 22:11:46.987892 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:46.987860    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:47.392265 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:47.392225    4460 patch_prober.go:29] interesting pod/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 22:11:47.392664 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:47.392306    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x" podUID="60161a74-1f84-4f09-9a66-b0c1d1fdce14" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 22:11:47.469579 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:47.469544    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:48.248212 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:48.248178    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:49.928192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:49.928153    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:50.513808 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:50.513772    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:51.160045 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:51.159143    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 22:11:51.453895 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:11:51.453811    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:11:51.552669 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:51.552628    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:52.340036 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:52.339998    4460 generic.go:356] "Generic (PLEG): container finished" podID="60161a74-1f84-4f09-9a66-b0c1d1fdce14" containerID="03b7e02788741058f0b69f5b5a10c4ca570ec8f5a6bd65e4df21e02fe9b83f59" exitCode=0
Oct 31 22:11:52.399832 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:52.397591    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x" event={"ID":"60161a74-1f84-4f09-9a66-b0c1d1fdce14","Type":"ContainerDied","Data":"03b7e02788741058f0b69f5b5a10c4ca570ec8f5a6bd65e4df21e02fe9b83f59"}
Oct 31 22:11:52.441038 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:52.440994    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"]
Oct 31 22:11:52.455870 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:52.455837    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 22:11:52.550670 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:52.550637    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-workload-certs\") pod \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") "
Oct 31 22:11:52.550880 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:52.550690    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istio-data\") pod \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") "
Oct 31 22:11:52.550880 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:52.550735    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/60161a74-1f84-4f09-9a66-b0c1d1fdce14-kube-api-access-q5qhh\" (UniqueName: \"kubernetes.io/projected/60161a74-1f84-4f09-9a66-b0c1d1fdce14-kube-api-access-q5qhh\") pod \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") "
Oct 31 22:11:52.550880 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:52.550803    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istio-envoy\") pod \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") "
Oct 31 22:11:52.550880 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:52.550829    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istio-token\" (UniqueName: \"kubernetes.io/projected/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istio-token\") pod \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") "
Oct 31 22:11:52.551182 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:52.550993    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-workload-certs" pod "60161a74-1f84-4f09-9a66-b0c1d1fdce14" (UID: "60161a74-1f84-4f09-9a66-b0c1d1fdce14"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:11:52.551182 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:52.551067    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istio-podinfo\") pod \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") "
Oct 31 22:11:52.551182 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:52.551107    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istiod-ca-cert\") pod \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") "
Oct 31 22:11:52.551182 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:52.551161    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-credential-socket\") pod \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") "
Oct 31 22:11:52.551397 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:52.551201    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-workload-socket\") pod \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\" (UID: \"60161a74-1f84-4f09-9a66-b0c1d1fdce14\") "
Oct 31 22:11:52.551397 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:52.551065    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istio-data" pod "60161a74-1f84-4f09-9a66-b0c1d1fdce14" (UID: "60161a74-1f84-4f09-9a66-b0c1d1fdce14"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:11:52.551699 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:52.551673    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-workload-socket" pod "60161a74-1f84-4f09-9a66-b0c1d1fdce14" (UID: "60161a74-1f84-4f09-9a66-b0c1d1fdce14"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:11:52.551964 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:52.551913    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:11:52.552052 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:52.551968    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:11:52.552052 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:52.551984    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:11:52.552052 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:52.552001    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istiod-ca-cert" pod "60161a74-1f84-4f09-9a66-b0c1d1fdce14" (UID: "60161a74-1f84-4f09-9a66-b0c1d1fdce14"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:11:52.552221 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:52.552141    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-credential-socket" pod "60161a74-1f84-4f09-9a66-b0c1d1fdce14" (UID: "60161a74-1f84-4f09-9a66-b0c1d1fdce14"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:11:52.554030 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:52.553992    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istio-podinfo" pod "60161a74-1f84-4f09-9a66-b0c1d1fdce14" (UID: "60161a74-1f84-4f09-9a66-b0c1d1fdce14"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 22:11:52.554451 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:52.554421    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istio-envoy" pod "60161a74-1f84-4f09-9a66-b0c1d1fdce14" (UID: "60161a74-1f84-4f09-9a66-b0c1d1fdce14"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:11:52.556151 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:52.556116    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60161a74-1f84-4f09-9a66-b0c1d1fdce14-kube-api-access-q5qhh" pod "60161a74-1f84-4f09-9a66-b0c1d1fdce14" (UID: "60161a74-1f84-4f09-9a66-b0c1d1fdce14"). InnerVolumeSpecName "kube-api-access-q5qhh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:11:52.557154 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:52.557128    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istio-token" pod "60161a74-1f84-4f09-9a66-b0c1d1fdce14" (UID: "60161a74-1f84-4f09-9a66-b0c1d1fdce14"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:11:52.565174 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:52.564997    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:52.653891 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:52.653790    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:11:52.653891 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:52.653827    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:11:52.653891 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:52.653842    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:11:52.653891 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:52.653856    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60161a74-1f84-4f09-9a66-b0c1d1fdce14-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:11:52.653891 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:52.653874    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60161a74-1f84-4f09-9a66-b0c1d1fdce14-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:11:52.653891 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:52.653891    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5qhh\" (UniqueName: \"kubernetes.io/projected/60161a74-1f84-4f09-9a66-b0c1d1fdce14-kube-api-access-q5qhh\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:11:53.325350 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:53.325297    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:53.354612 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:53.354547    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x" event={"ID":"60161a74-1f84-4f09-9a66-b0c1d1fdce14","Type":"ContainerDied","Data":"be6b7dec75dacfb88de7f99fcdbaf5c8af1caa0f277078c0165b27af6b77c01b"}
Oct 31 22:11:53.355186 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:53.354644    4460 scope.go:122] "RemoveContainer" containerID="03b7e02788741058f0b69f5b5a10c4ca570ec8f5a6bd65e4df21e02fe9b83f59"
Oct 31 22:11:53.355186 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:53.354880    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"
Oct 31 22:11:53.475075 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:53.475005    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"]
Oct 31 22:11:53.488539 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:53.488510    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-5237cd-openshift-default-5f58b4f7bf-6hr4x"]
Oct 31 22:11:54.226734 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:54.226699    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="60161a74-1f84-4f09-9a66-b0c1d1fdce14" path="/var/lib/kubelet/pods/60161a74-1f84-4f09-9a66-b0c1d1fdce14/volumes"
Oct 31 22:11:55.006618 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:55.006573    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:56.698109 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:56.698072    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:58.253876 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:58.253837    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-4b5989-7bdb89b88c-qgl9q"]
Oct 31 22:11:58.370664 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:58.370621    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:11:58.519025 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:11:58.516486    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:00.112560 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:00.112527    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:01.682829 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:01.682789    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:01.809290 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:01.809252    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:03.254330 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:03.254289    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b5989-7bdb89b88c-qgl9q" podUID="4c072735-3c6f-4c6a-9d51-b535fe7f4dfe" containerName="payload-pre-processing" containerID="cri-o://daf570975ab6fb4180d7ddf1c97dc9cb1d678e01954f556c0018426b99defd2f" gracePeriod=25
Oct 31 22:12:03.535261 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:03.535228    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:05.408992 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:05.408951    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:07.102830 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:07.102792    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:07.478536 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:07.478439    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" podUID="af7e0e0c-cf52-4270-b528-35718339996c" containerName="main" containerID="cri-o://dc13d5467c37ab13f3d4d85e18efb588620e00e8261274c81f0a724b8fc5049f" gracePeriod=45
Oct 31 22:12:07.769793 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:07.769764    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 22:12:07.867884 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:07.867850    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af7e0e0c-cf52-4270-b528-35718339996c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/af7e0e0c-cf52-4270-b528-35718339996c-dshm\") pod \"af7e0e0c-cf52-4270-b528-35718339996c\" (UID: \"af7e0e0c-cf52-4270-b528-35718339996c\") "
Oct 31 22:12:07.868122 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:07.868013    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af7e0e0c-cf52-4270-b528-35718339996c-kube-api-access-kg8kr\" (UniqueName: \"kubernetes.io/projected/af7e0e0c-cf52-4270-b528-35718339996c-kube-api-access-kg8kr\") pod \"af7e0e0c-cf52-4270-b528-35718339996c\" (UID: \"af7e0e0c-cf52-4270-b528-35718339996c\") "
Oct 31 22:12:07.868204 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:07.868119    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/af7e0e0c-cf52-4270-b528-35718339996c-tls-certs\" (UniqueName: \"kubernetes.io/secret/af7e0e0c-cf52-4270-b528-35718339996c-tls-certs\") pod \"af7e0e0c-cf52-4270-b528-35718339996c\" (UID: \"af7e0e0c-cf52-4270-b528-35718339996c\") "
Oct 31 22:12:07.868273 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:07.868205    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af7e0e0c-cf52-4270-b528-35718339996c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af7e0e0c-cf52-4270-b528-35718339996c-model-cache\") pod \"af7e0e0c-cf52-4270-b528-35718339996c\" (UID: \"af7e0e0c-cf52-4270-b528-35718339996c\") "
Oct 31 22:12:07.868382 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:07.868360    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af7e0e0c-cf52-4270-b528-35718339996c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af7e0e0c-cf52-4270-b528-35718339996c-tmp-dir\") pod \"af7e0e0c-cf52-4270-b528-35718339996c\" (UID: \"af7e0e0c-cf52-4270-b528-35718339996c\") "
Oct 31 22:12:07.868455 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:07.868432    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af7e0e0c-cf52-4270-b528-35718339996c-home\" (UniqueName: \"kubernetes.io/empty-dir/af7e0e0c-cf52-4270-b528-35718339996c-home\") pod \"af7e0e0c-cf52-4270-b528-35718339996c\" (UID: \"af7e0e0c-cf52-4270-b528-35718339996c\") "
Oct 31 22:12:07.868592 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:07.868571    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af7e0e0c-cf52-4270-b528-35718339996c-model-cache" pod "af7e0e0c-cf52-4270-b528-35718339996c" (UID: "af7e0e0c-cf52-4270-b528-35718339996c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:12:07.868757 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:07.868729    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af7e0e0c-cf52-4270-b528-35718339996c-tmp-dir" pod "af7e0e0c-cf52-4270-b528-35718339996c" (UID: "af7e0e0c-cf52-4270-b528-35718339996c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:12:07.868864 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:07.868838    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af7e0e0c-cf52-4270-b528-35718339996c-home" pod "af7e0e0c-cf52-4270-b528-35718339996c" (UID: "af7e0e0c-cf52-4270-b528-35718339996c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:12:07.869299 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:07.869253    4460 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af7e0e0c-cf52-4270-b528-35718339996c-model-cache\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:12:07.869299 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:07.869273    4460 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af7e0e0c-cf52-4270-b528-35718339996c-tmp-dir\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:12:07.869299 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:07.869281    4460 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af7e0e0c-cf52-4270-b528-35718339996c-home\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:12:07.871990 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:07.871956    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af7e0e0c-cf52-4270-b528-35718339996c-dshm" pod "af7e0e0c-cf52-4270-b528-35718339996c" (UID: "af7e0e0c-cf52-4270-b528-35718339996c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:12:07.872207 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:07.872181    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af7e0e0c-cf52-4270-b528-35718339996c-kube-api-access-kg8kr" pod "af7e0e0c-cf52-4270-b528-35718339996c" (UID: "af7e0e0c-cf52-4270-b528-35718339996c"). InnerVolumeSpecName "kube-api-access-kg8kr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:12:07.873296 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:07.873258    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af7e0e0c-cf52-4270-b528-35718339996c-tls-certs" pod "af7e0e0c-cf52-4270-b528-35718339996c" (UID: "af7e0e0c-cf52-4270-b528-35718339996c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 22:12:07.970350 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:07.970313    4460 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af7e0e0c-cf52-4270-b528-35718339996c-tls-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:12:07.970350 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:07.970345    4460 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af7e0e0c-cf52-4270-b528-35718339996c-dshm\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:12:07.970350 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:07.970357    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kg8kr\" (UniqueName: \"kubernetes.io/projected/af7e0e0c-cf52-4270-b528-35718339996c-kube-api-access-kg8kr\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:12:08.285873 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:08.285838    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b5989-7bdb89b88c-qgl9q" podUID="4c072735-3c6f-4c6a-9d51-b535fe7f4dfe" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:12:08.469268 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:08.469219    4460 generic.go:356] "Generic (PLEG): container finished" podID="af7e0e0c-cf52-4270-b528-35718339996c" containerID="dc13d5467c37ab13f3d4d85e18efb588620e00e8261274c81f0a724b8fc5049f" exitCode=0
Oct 31 22:12:08.469460 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:08.469273    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"
Oct 31 22:12:08.469460 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:08.469294    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" event={"ID":"af7e0e0c-cf52-4270-b528-35718339996c","Type":"ContainerDied","Data":"dc13d5467c37ab13f3d4d85e18efb588620e00e8261274c81f0a724b8fc5049f"}
Oct 31 22:12:08.469460 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:08.469337    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274" event={"ID":"af7e0e0c-cf52-4270-b528-35718339996c","Type":"ContainerDied","Data":"5158e2d12f6ff9616d3df107159a399424fe3f738dfa0c70a80721dff32b0fb7"}
Oct 31 22:12:08.469460 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:08.469356    4460 scope.go:122] "RemoveContainer" containerID="dc13d5467c37ab13f3d4d85e18efb588620e00e8261274c81f0a724b8fc5049f"
Oct 31 22:12:08.481358 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:08.480998    4460 scope.go:122] "RemoveContainer" containerID="dc13d5467c37ab13f3d4d85e18efb588620e00e8261274c81f0a724b8fc5049f"
Oct 31 22:12:08.481442 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:12:08.481378    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc13d5467c37ab13f3d4d85e18efb588620e00e8261274c81f0a724b8fc5049f\": container with ID starting with dc13d5467c37ab13f3d4d85e18efb588620e00e8261274c81f0a724b8fc5049f not found: ID does not exist" containerID="dc13d5467c37ab13f3d4d85e18efb588620e00e8261274c81f0a724b8fc5049f"
Oct 31 22:12:08.481442 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:08.481422    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc13d5467c37ab13f3d4d85e18efb588620e00e8261274c81f0a724b8fc5049f"} err="failed to get container status \"dc13d5467c37ab13f3d4d85e18efb588620e00e8261274c81f0a724b8fc5049f\": rpc error: code = NotFound desc = could not find container \"dc13d5467c37ab13f3d4d85e18efb588620e00e8261274c81f0a724b8fc5049f\": container with ID starting with dc13d5467c37ab13f3d4d85e18efb588620e00e8261274c81f0a724b8fc5049f not found: ID does not exist"
Oct 31 22:12:08.530416 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:08.530380    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"]
Oct 31 22:12:08.543351 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:08.543316    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-4b5989/distinct-2-w0-4b5989-kserve-6cd6ff5b57-jv274"]
Oct 31 22:12:08.837457 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:08.837364    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:10.228406 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:10.228369    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="af7e0e0c-cf52-4270-b528-35718339996c" path="/var/lib/kubelet/pods/af7e0e0c-cf52-4270-b528-35718339996c/volumes"
Oct 31 22:12:10.517796 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:10.517740    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:12.113181 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:12.113132    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:13.743734 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:13.743631    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:15.436806 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:15.436768    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:17.149249 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:17.149210    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:18.286359 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:18.286315    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b5989-7bdb89b88c-qgl9q" podUID="4c072735-3c6f-4c6a-9d51-b535fe7f4dfe" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:12:18.906702 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:18.906659    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:20.510190 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:20.510128    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:21.454868 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:12:21.454828    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:12:21.756584 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:21.756489    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:22.329296 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:22.329261    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:23.571596 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:23.571561    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b5989-7bdb89b88c-qgl9q"
Oct 31 22:12:23.628268 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:23.628237    4460 generic.go:356] "Generic (PLEG): container finished" podID="4c072735-3c6f-4c6a-9d51-b535fe7f4dfe" containerID="daf570975ab6fb4180d7ddf1c97dc9cb1d678e01954f556c0018426b99defd2f" exitCode=0
Oct 31 22:12:23.628461 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:23.628283    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b5989-7bdb89b88c-qgl9q"
Oct 31 22:12:23.628461 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:23.628313    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b5989-7bdb89b88c-qgl9q" event={"ID":"4c072735-3c6f-4c6a-9d51-b535fe7f4dfe","Type":"ContainerDied","Data":"daf570975ab6fb4180d7ddf1c97dc9cb1d678e01954f556c0018426b99defd2f"}
Oct 31 22:12:23.628461 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:23.628376    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-4b5989-7bdb89b88c-qgl9q" event={"ID":"4c072735-3c6f-4c6a-9d51-b535fe7f4dfe","Type":"ContainerDied","Data":"1717acd18afc1aca63cb805b9910f3721aec96da4863bd78f192e25471d00968"}
Oct 31 22:12:23.628461 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:23.628401    4460 scope.go:122] "RemoveContainer" containerID="daf570975ab6fb4180d7ddf1c97dc9cb1d678e01954f556c0018426b99defd2f"
Oct 31 22:12:23.640510 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:23.640484    4460 scope.go:122] "RemoveContainer" containerID="daf570975ab6fb4180d7ddf1c97dc9cb1d678e01954f556c0018426b99defd2f"
Oct 31 22:12:23.640860 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:12:23.640835    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"daf570975ab6fb4180d7ddf1c97dc9cb1d678e01954f556c0018426b99defd2f\": container with ID starting with daf570975ab6fb4180d7ddf1c97dc9cb1d678e01954f556c0018426b99defd2f not found: ID does not exist" containerID="daf570975ab6fb4180d7ddf1c97dc9cb1d678e01954f556c0018426b99defd2f"
Oct 31 22:12:23.640952 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:23.640869    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daf570975ab6fb4180d7ddf1c97dc9cb1d678e01954f556c0018426b99defd2f"} err="failed to get container status \"daf570975ab6fb4180d7ddf1c97dc9cb1d678e01954f556c0018426b99defd2f\": rpc error: code = NotFound desc = could not find container \"daf570975ab6fb4180d7ddf1c97dc9cb1d678e01954f556c0018426b99defd2f\": container with ID starting with daf570975ab6fb4180d7ddf1c97dc9cb1d678e01954f556c0018426b99defd2f not found: ID does not exist"
Oct 31 22:12:23.680474 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:23.680436    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4c072735-3c6f-4c6a-9d51-b535fe7f4dfe-kube-api-access-4pmpd\" (UniqueName: \"kubernetes.io/projected/4c072735-3c6f-4c6a-9d51-b535fe7f4dfe-kube-api-access-4pmpd\") pod \"4c072735-3c6f-4c6a-9d51-b535fe7f4dfe\" (UID: \"4c072735-3c6f-4c6a-9d51-b535fe7f4dfe\") "
Oct 31 22:12:23.680651 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:23.680541    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4c072735-3c6f-4c6a-9d51-b535fe7f4dfe-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c072735-3c6f-4c6a-9d51-b535fe7f4dfe-plugins-config-volume\") pod \"4c072735-3c6f-4c6a-9d51-b535fe7f4dfe\" (UID: \"4c072735-3c6f-4c6a-9d51-b535fe7f4dfe\") "
Oct 31 22:12:23.681175 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:23.681133    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c072735-3c6f-4c6a-9d51-b535fe7f4dfe-plugins-config-volume" pod "4c072735-3c6f-4c6a-9d51-b535fe7f4dfe" (UID: "4c072735-3c6f-4c6a-9d51-b535fe7f4dfe"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:12:23.683169 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:23.683134    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c072735-3c6f-4c6a-9d51-b535fe7f4dfe-kube-api-access-4pmpd" pod "4c072735-3c6f-4c6a-9d51-b535fe7f4dfe" (UID: "4c072735-3c6f-4c6a-9d51-b535fe7f4dfe"). InnerVolumeSpecName "kube-api-access-4pmpd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:12:23.782464 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:23.782373    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4pmpd\" (UniqueName: \"kubernetes.io/projected/4c072735-3c6f-4c6a-9d51-b535fe7f4dfe-kube-api-access-4pmpd\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:12:23.782464 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:23.782415    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c072735-3c6f-4c6a-9d51-b535fe7f4dfe-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:12:23.828484 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:23.828433    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:23.981196 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:23.981165    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-4b5989-7bdb89b88c-qgl9q"]
Oct 31 22:12:23.984974 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:23.984917    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-4b5989-7bdb89b88c-qgl9q"]
Oct 31 22:12:24.008054 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:24.008008    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:24.228075 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:24.227990    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4c072735-3c6f-4c6a-9d51-b535fe7f4dfe" path="/var/lib/kubelet/pods/4c072735-3c6f-4c6a-9d51-b535fe7f4dfe/volumes"
Oct 31 22:12:25.751933 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:25.751894    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:27.519655 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:27.519619    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:28.537963 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:28.534204    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 22:12:29.316271 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:29.316217    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:30.613786 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:30.613239    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:30.723749 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:30.723712    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:30.928391 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:30.928282    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:31.083334 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:31.083294    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:32.704897 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:32.704858    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:34.313232 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:34.313194    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:36.002405 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:36.002368    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:37.528913 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:37.528871    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:37.660964 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:37.660204    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:39.305425 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:39.305369    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:40.989767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:40.989731    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:42.670769 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:42.670733    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:44.395518 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:44.395473    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:45.244554 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:45.244514    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:46.308710 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:46.308675    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:47.375389 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:47.375347    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:47.855187 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:47.854976    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:48.112746 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.112668    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:48.231155 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.231122    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"]
Oct 31 22:12:48.232290 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.232270    4460 cpu_manager.go:414] "containerMap: removing container" podUID="60161a74-1f84-4f09-9a66-b0c1d1fdce14" containerName="istio-proxy"
Oct 31 22:12:48.232349 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.232293    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="60161a74-1f84-4f09-9a66-b0c1d1fdce14" containerName="istio-proxy"
Oct 31 22:12:48.232349 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.232315    4460 cpu_manager.go:414] "containerMap: removing container" podUID="af7e0e0c-cf52-4270-b528-35718339996c" containerName="main"
Oct 31 22:12:48.232349 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.232323    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="af7e0e0c-cf52-4270-b528-35718339996c" containerName="main"
Oct 31 22:12:48.232462 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.232373    4460 cpu_manager.go:414] "containerMap: removing container" podUID="4c072735-3c6f-4c6a-9d51-b535fe7f4dfe" containerName="payload-pre-processing"
Oct 31 22:12:48.232462 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.232381    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4c072735-3c6f-4c6a-9d51-b535fe7f4dfe" containerName="payload-pre-processing"
Oct 31 22:12:48.232577 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.232566    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="60161a74-1f84-4f09-9a66-b0c1d1fdce14" containerName="istio-proxy"
Oct 31 22:12:48.232624 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.232583    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="af7e0e0c-cf52-4270-b528-35718339996c" containerName="main"
Oct 31 22:12:48.232624 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.232599    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="4c072735-3c6f-4c6a-9d51-b535fe7f4dfe" containerName="payload-pre-processing"
Oct 31 22:12:48.237266 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.237240    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:12:48.255169 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.255131    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"]
Oct 31 22:12:48.282617 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.282551    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:48.314737 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.314698    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-5c54fc-openshift-default-dockercfg-f6wgk\""
Oct 31 22:12:48.360455 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.360420    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-credential-socket\") pod \"e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") " pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:12:48.360619 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.360485    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-workload-certs\") pod \"e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") " pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:12:48.360619 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.360558    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-istio-envoy\") pod \"e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") " pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:12:48.360619 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.360613    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/390c66e4-91b0-4110-92c3-fd754193823d-istio-podinfo\") pod \"e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") " pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:12:48.360764 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.360741    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/390c66e4-91b0-4110-92c3-fd754193823d-istio-token\") pod \"e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") " pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:12:48.360862 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.360848    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/390c66e4-91b0-4110-92c3-fd754193823d-istiod-ca-cert\") pod \"e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") " pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:12:48.360900 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.360881    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlpjb\" (UniqueName: \"kubernetes.io/projected/390c66e4-91b0-4110-92c3-fd754193823d-kube-api-access-jlpjb\") pod \"e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") " pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:12:48.360935 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.360905    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-workload-socket\") pod \"e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") " pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:12:48.361048 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.361035    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-istio-data\") pod \"e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") " pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:12:48.405498 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.405398    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:48.471963 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.471791    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-istio-data\") pod \"e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") " pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:12:48.471963 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.471911    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-credential-socket\") pod \"e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") " pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:12:48.472237 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.471992    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-workload-certs\") pod \"e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") " pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:12:48.472237 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.472054    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-istio-envoy\") pod \"e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") " pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:12:48.472237 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.472129    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/390c66e4-91b0-4110-92c3-fd754193823d-istio-podinfo\") pod \"e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") " pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:12:48.472237 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.472214    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/390c66e4-91b0-4110-92c3-fd754193823d-istio-token\") pod \"e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") " pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:12:48.472478 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.472301    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/390c66e4-91b0-4110-92c3-fd754193823d-istiod-ca-cert\") pod \"e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") " pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:12:48.472478 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.472337    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlpjb\" (UniqueName: \"kubernetes.io/projected/390c66e4-91b0-4110-92c3-fd754193823d-kube-api-access-jlpjb\") pod \"e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") " pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:12:48.472478 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.472369    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-workload-socket\") pod \"e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") " pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:12:48.487964 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.472887    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-workload-socket\") pod \"e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") " pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:12:48.487964 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.473180    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-istio-data\") pod \"e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") " pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:12:48.487964 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.473413    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-credential-socket\") pod \"e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") " pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:12:48.487964 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.473635    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-workload-certs\") pod \"e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") " pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:12:48.487964 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.475680    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/390c66e4-91b0-4110-92c3-fd754193823d-istiod-ca-cert\") pod \"e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") " pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:12:48.509452 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.509411    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/390c66e4-91b0-4110-92c3-fd754193823d-istio-podinfo\") pod \"e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") " pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:12:48.510332 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.510305    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-istio-envoy\") pod \"e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") " pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:12:48.528960 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.525334    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlpjb\" (UniqueName: \"kubernetes.io/projected/390c66e4-91b0-4110-92c3-fd754193823d-kube-api-access-jlpjb\") pod \"e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") " pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:12:48.563985 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.563447    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/390c66e4-91b0-4110-92c3-fd754193823d-istio-token\") pod \"e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") " pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:12:48.563985 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.563757    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:12:48.969809 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.969775    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"]
Oct 31 22:12:48.971352 ip-10-0-136-157 kubenswrapper[4460]: W1031 22:12:48.971315    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod390c66e4_91b0_4110_92c3_fd754193823d.slice/crio-716129c29fb41f41ac23fee03580ffc51042037bca7b9629595eff7ce7f743b9 WatchSource:0}: Error finding container 716129c29fb41f41ac23fee03580ffc51042037bca7b9629595eff7ce7f743b9: Status 404 returned error can't find the container with id 716129c29fb41f41ac23fee03580ffc51042037bca7b9629595eff7ce7f743b9
Oct 31 22:12:48.974897 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.974856    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:12:48.975035 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.974966    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:12:48.975035 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:48.975018    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:12:49.876195 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:49.876157    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv" event={"ID":"390c66e4-91b0-4110-92c3-fd754193823d","Type":"ContainerStarted","Data":"042625313bc5ef40a0e4e1d1edf6965ed38f3a1ca719a93ef72c1422edacb6e2"}
Oct 31 22:12:49.876607 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:49.876217    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv" event={"ID":"390c66e4-91b0-4110-92c3-fd754193823d","Type":"ContainerStarted","Data":"716129c29fb41f41ac23fee03580ffc51042037bca7b9629595eff7ce7f743b9"}
Oct 31 22:12:49.933304 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:49.933270    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:49.957548 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:49.957499    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv" podStartSLOduration=1.957485044 podStartE2EDuration="1.957485044s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:12:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:12:49.943294377 +0000 UTC m=+3409.859603808" watchObservedRunningTime="2026-10-31 22:12:49.957485044 +0000 UTC m=+3409.873794475"
Oct 31 22:12:50.564613 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:50.564579    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:12:50.570840 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:50.570813    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:12:50.635914 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:50.635881    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:50.884033 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:50.883949    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:12:50.885137 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:50.885113    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:12:51.454621 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:12:51.454564    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:12:51.713364 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:51.713277    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:52.282517 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:52.282472    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:52.399682 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:52.399650    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:53.570644 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:53.570602    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:53.701406 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:53.701357    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:53.837567 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:53.837465    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:54.205414 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:54.205318    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:54.331774 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:54.331738    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:54.366209 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:54.366175    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5c54fc-7467846795-qndhq"]
Oct 31 22:12:54.371669 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:54.371649    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5c54fc-7467846795-qndhq"
Oct 31 22:12:54.388432 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:54.388405    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5c54fc-7467846795-qndhq"]
Oct 31 22:12:54.396823 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:54.396800    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-5c54fc\""
Oct 31 22:12:54.397016 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:54.396862    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-5c54fc-dockercfg-scq4b\""
Oct 31 22:12:54.418594 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:54.418558    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7d48a8c-34fc-4974-8cc6-1346a27fcc65-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-5c54fc-7467846795-qndhq\" (UID: \"b7d48a8c-34fc-4974-8cc6-1346a27fcc65\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5c54fc-7467846795-qndhq"
Oct 31 22:12:54.418912 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:54.418889    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsgmm\" (UniqueName: \"kubernetes.io/projected/b7d48a8c-34fc-4974-8cc6-1346a27fcc65-kube-api-access-tsgmm\") pod \"payload-pre-processing-e2e-inf-b-5c54fc-7467846795-qndhq\" (UID: \"b7d48a8c-34fc-4974-8cc6-1346a27fcc65\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5c54fc-7467846795-qndhq"
Oct 31 22:12:54.657740 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:54.657698    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsgmm\" (UniqueName: \"kubernetes.io/projected/b7d48a8c-34fc-4974-8cc6-1346a27fcc65-kube-api-access-tsgmm\") pod \"payload-pre-processing-e2e-inf-b-5c54fc-7467846795-qndhq\" (UID: \"b7d48a8c-34fc-4974-8cc6-1346a27fcc65\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5c54fc-7467846795-qndhq"
Oct 31 22:12:54.658856 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:54.658825    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7d48a8c-34fc-4974-8cc6-1346a27fcc65-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-5c54fc-7467846795-qndhq\" (UID: \"b7d48a8c-34fc-4974-8cc6-1346a27fcc65\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5c54fc-7467846795-qndhq"
Oct 31 22:12:54.660127 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:54.660019    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7d48a8c-34fc-4974-8cc6-1346a27fcc65-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-5c54fc-7467846795-qndhq\" (UID: \"b7d48a8c-34fc-4974-8cc6-1346a27fcc65\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5c54fc-7467846795-qndhq"
Oct 31 22:12:54.707163 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:54.707124    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsgmm\" (UniqueName: \"kubernetes.io/projected/b7d48a8c-34fc-4974-8cc6-1346a27fcc65-kube-api-access-tsgmm\") pod \"payload-pre-processing-e2e-inf-b-5c54fc-7467846795-qndhq\" (UID: \"b7d48a8c-34fc-4974-8cc6-1346a27fcc65\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5c54fc-7467846795-qndhq"
Oct 31 22:12:54.989544 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:54.989464    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5c54fc-7467846795-qndhq"
Oct 31 22:12:55.297127 ip-10-0-136-157 kubenswrapper[4460]: W1031 22:12:55.297093    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7d48a8c_34fc_4974_8cc6_1346a27fcc65.slice/crio-3230ed0a1685d69410467e9f88690da086dba0adfa7a956f8b8f139655249e71 WatchSource:0}: Error finding container 3230ed0a1685d69410467e9f88690da086dba0adfa7a956f8b8f139655249e71: Status 404 returned error can't find the container with id 3230ed0a1685d69410467e9f88690da086dba0adfa7a956f8b8f139655249e71
Oct 31 22:12:55.297314 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:55.297291    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5c54fc-7467846795-qndhq"]
Oct 31 22:12:55.643777 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:55.643679    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:55.924068 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:55.923970    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5c54fc-7467846795-qndhq" event={"ID":"b7d48a8c-34fc-4974-8cc6-1346a27fcc65","Type":"ContainerStarted","Data":"676db78d3a27c5c5a346f580ff3bce56d0e4bf6e3acd048cbfd30ac114d09951"}
Oct 31 22:12:55.924068 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:55.924059    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5c54fc-7467846795-qndhq" event={"ID":"b7d48a8c-34fc-4974-8cc6-1346a27fcc65","Type":"ContainerStarted","Data":"3230ed0a1685d69410467e9f88690da086dba0adfa7a956f8b8f139655249e71"}
Oct 31 22:12:55.924487 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:55.924082    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5c54fc-7467846795-qndhq"
Oct 31 22:12:55.970198 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:55.970094    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5c54fc-7467846795-qndhq" podStartSLOduration=1.9700746850000002 podStartE2EDuration="1.970074685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:12:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:12:55.968299401 +0000 UTC m=+3415.884608831" watchObservedRunningTime="2026-10-31 22:12:55.970074685 +0000 UTC m=+3415.886384116"
Oct 31 22:12:57.317639 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:57.317601    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:12:58.888153 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:12:58.888111    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:00.505351 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:00.505316    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:02.131273 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:02.131235    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:03.810467 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:03.810424    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:05.182058 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:05.181972    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 22:13:05.439799 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:05.439712    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:06.935521 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:06.935488    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5c54fc-7467846795-qndhq"
Oct 31 22:13:07.221970 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:07.221871    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:08.737765 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:08.737729    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:09.316619 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:09.316571    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:09.426053 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:09.426017    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:09.538571 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:09.538534    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:09.918249 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:09.918213    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:11.211500 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:11.211464    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:12.904076 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:12.904031    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:14.529266 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:14.529231    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:16.208333 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:16.208300    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:16.332488 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:16.331560    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:16.596875 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:16.596841    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:17.909479 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:17.909441    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:19.538188 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:19.538146    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:21.119074 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:21.119027    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:21.453701 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:13:21.453617    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:13:22.776217 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:22.776165    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:24.564307 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:24.564271    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:26.226511 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:26.226473    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:27.905876 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:27.905838    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:29.505614 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:29.505567    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:31.083325 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:31.083260    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:32.775861 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:32.775828    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:32.898964 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:32.898213    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:33.014055 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:33.014021    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:34.528921 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:34.528863    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:36.289425 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:36.289374    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:37.217382 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:37.217344    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:37.327119 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:37.327082    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:38.017579 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:38.017532    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:39.734977 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:39.734899    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:41.408503 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:41.408463    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:42.995525 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:42.995432    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:44.602823 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:44.602783    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:46.236309 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:46.236268    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:47.886642 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:47.886601    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:49.516807 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:49.516760    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:49.638312 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:49.638270    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:49.752075 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:49.752018    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:50.264056 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:50.264013    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:50.583755 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:50.583650    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:50.765571 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:50.765529    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:51.304955 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:51.304901    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:51.456504 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:13:51.456469    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:13:52.921171 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:52.921137    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:54.621006 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:54.620963    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:55.161807 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:55.161766    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 22:13:56.268992 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:56.268953    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:56.600013 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:56.599896    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:56.737847 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:56.737804    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:58.012360 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:58.012319    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:13:59.660299 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:13:59.660260    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:01.346966 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:01.343598    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:03.025690 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:03.025646    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:04.627195 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:04.627156    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:05.798822 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:05.798793    4460 generic.go:356] "Generic (PLEG): container finished" podID="390c66e4-91b0-4110-92c3-fd754193823d" containerID="042625313bc5ef40a0e4e1d1edf6965ed38f3a1ca719a93ef72c1422edacb6e2" exitCode=0
Oct 31 22:14:05.799215 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:05.798873    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv" event={"ID":"390c66e4-91b0-4110-92c3-fd754193823d","Type":"ContainerDied","Data":"042625313bc5ef40a0e4e1d1edf6965ed38f3a1ca719a93ef72c1422edacb6e2"}
Oct 31 22:14:05.799382 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:05.799366    4460 scope.go:122] "RemoveContainer" containerID="042625313bc5ef40a0e4e1d1edf6965ed38f3a1ca719a93ef72c1422edacb6e2"
Oct 31 22:14:05.801292 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:05.801260    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:14:05.801405 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:05.801345    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:14:05.801405 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:05.801395    4460 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 22:14:05.884794 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:05.884762    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:14:06.318473 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:06.318440    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:06.808203 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:06.808168    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv" event={"ID":"390c66e4-91b0-4110-92c3-fd754193823d","Type":"ContainerStarted","Data":"4e95ab6cd2ec240baa1145ea1a369aee448ed29a08a3733be044b148403b9485"}
Oct 31 22:14:07.502412 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:07.502370    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:07.565266 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:07.565216    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:14:07.566896 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:07.566865    4460 patch_prober.go:29] interesting pod/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.220:15021/healthz/ready\": dial tcp 10.133.0.220:15021: connect: connection refused" start-of-body=
Oct 31 22:14:07.567061 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:07.566992    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv" podUID="390c66e4-91b0-4110-92c3-fd754193823d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.220:15021/healthz/ready\": dial tcp 10.133.0.220:15021: connect: connection refused"
Oct 31 22:14:07.625891 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:07.625854    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:08.155011 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:08.154972    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:08.565585 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:08.565542    4460 patch_prober.go:29] interesting pod/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.220:15021/healthz/ready\": dial tcp 10.133.0.220:15021: connect: connection refused" start-of-body=
Oct 31 22:14:08.565779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:08.565619    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv" podUID="390c66e4-91b0-4110-92c3-fd754193823d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.220:15021/healthz/ready\": dial tcp 10.133.0.220:15021: connect: connection refused"
Oct 31 22:14:09.564988 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:09.564896    4460 patch_prober.go:29] interesting pod/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.220:15021/healthz/ready\": dial tcp 10.133.0.220:15021: connect: connection refused" start-of-body=
Oct 31 22:14:09.564988 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:09.564970    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv" podUID="390c66e4-91b0-4110-92c3-fd754193823d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.220:15021/healthz/ready\": dial tcp 10.133.0.220:15021: connect: connection refused"
Oct 31 22:14:09.845775 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:09.845665    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:10.565300 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:10.565264    4460 patch_prober.go:29] interesting pod/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.220:15021/healthz/ready\": dial tcp 10.133.0.220:15021: connect: connection refused" start-of-body=
Oct 31 22:14:10.565851 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:10.565321    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv" podUID="390c66e4-91b0-4110-92c3-fd754193823d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.220:15021/healthz/ready\": dial tcp 10.133.0.220:15021: connect: connection refused"
Oct 31 22:14:11.565363 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:11.565019    4460 patch_prober.go:29] interesting pod/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.220:15021/healthz/ready\": dial tcp 10.133.0.220:15021: connect: connection refused" start-of-body=
Oct 31 22:14:11.565363 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:11.565072    4460 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv" podUID="390c66e4-91b0-4110-92c3-fd754193823d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.220:15021/healthz/ready\": dial tcp 10.133.0.220:15021: connect: connection refused"
Oct 31 22:14:11.604794 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:11.604750    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:12.569912 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:12.569877    4460 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:14:12.570331 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:12.570298    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:14:12.570849 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:12.570828    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:14:13.218065 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:13.218033    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:14.951148 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:14.951111    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:16.857055 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:16.857015    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:18.837985 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:18.837932    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:18.958547 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:18.958492    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:20.736198 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:20.736161    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:21.417949 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:21.417899    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:21.454372 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:14:21.454334    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:14:22.602793 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:22.602761    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:24.355567 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:24.355530    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:24.546197 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:24.546159    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 22:14:26.154607 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:26.154571    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:27.645452 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:27.645418    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:28.269194 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:28.269156    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:28.395141 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:28.395094    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:28.507652 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:28.507616    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:28.929058 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:28.929019    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:30.350452 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:30.350408    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:32.092596 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:32.092559    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:33.845495 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:33.845452    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:35.204780 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:35.204743    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:35.335308 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:35.335272    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:35.530581 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:35.530542    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"]
Oct 31 22:14:35.530810 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:35.530780    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns" podUID="1952bdcb-9771-45be-85ce-085a1c9df4f8" containerName="istio-proxy" containerID="cri-o://5be606e49fcbafc179b3b0af46223d6eaf056c8f0c0a59f45aa1508f00c19326" gracePeriod=30
Oct 31 22:14:35.575976 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:35.575923    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:35.685408 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:35.685308    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:37.445263 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:37.445226    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:37.836155 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:37.836116    4460 patch_prober.go:29] interesting pod/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 22:14:37.836340 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:37.836195    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns" podUID="1952bdcb-9771-45be-85ce-085a1c9df4f8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 22:14:39.264247 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:39.264209    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:40.814730 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:40.814671    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 22:14:40.893972 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:40.893921    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1952bdcb-9771-45be-85ce-085a1c9df4f8-istio-token\" (UniqueName: \"kubernetes.io/projected/1952bdcb-9771-45be-85ce-085a1c9df4f8-istio-token\") pod \"1952bdcb-9771-45be-85ce-085a1c9df4f8\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") "
Oct 31 22:14:40.894158 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:40.894043    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-workload-certs\") pod \"1952bdcb-9771-45be-85ce-085a1c9df4f8\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") "
Oct 31 22:14:40.894158 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:40.894091    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-credential-socket\") pod \"1952bdcb-9771-45be-85ce-085a1c9df4f8\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") "
Oct 31 22:14:40.894291 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:40.894167    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-istio-data\") pod \"1952bdcb-9771-45be-85ce-085a1c9df4f8\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") "
Oct 31 22:14:40.894351 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:40.894302    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1952bdcb-9771-45be-85ce-085a1c9df4f8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1952bdcb-9771-45be-85ce-085a1c9df4f8-istiod-ca-cert\") pod \"1952bdcb-9771-45be-85ce-085a1c9df4f8\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") "
Oct 31 22:14:40.894443 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:40.894425    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1952bdcb-9771-45be-85ce-085a1c9df4f8-kube-api-access-bbjd6\" (UniqueName: \"kubernetes.io/projected/1952bdcb-9771-45be-85ce-085a1c9df4f8-kube-api-access-bbjd6\") pod \"1952bdcb-9771-45be-85ce-085a1c9df4f8\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") "
Oct 31 22:14:40.894516 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:40.894473    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-workload-socket\") pod \"1952bdcb-9771-45be-85ce-085a1c9df4f8\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") "
Oct 31 22:14:40.894577 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:40.894527    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-istio-envoy\") pod \"1952bdcb-9771-45be-85ce-085a1c9df4f8\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") "
Oct 31 22:14:40.894638 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:40.894560    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-credential-socket" pod "1952bdcb-9771-45be-85ce-085a1c9df4f8" (UID: "1952bdcb-9771-45be-85ce-085a1c9df4f8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:14:40.894697 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:40.894648    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1952bdcb-9771-45be-85ce-085a1c9df4f8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1952bdcb-9771-45be-85ce-085a1c9df4f8-istio-podinfo\") pod \"1952bdcb-9771-45be-85ce-085a1c9df4f8\" (UID: \"1952bdcb-9771-45be-85ce-085a1c9df4f8\") "
Oct 31 22:14:40.895708 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:40.894788    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-workload-certs" pod "1952bdcb-9771-45be-85ce-085a1c9df4f8" (UID: "1952bdcb-9771-45be-85ce-085a1c9df4f8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:14:40.895708 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:40.894841    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-istio-data" pod "1952bdcb-9771-45be-85ce-085a1c9df4f8" (UID: "1952bdcb-9771-45be-85ce-085a1c9df4f8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:14:40.895708 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:40.895059    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-workload-socket" pod "1952bdcb-9771-45be-85ce-085a1c9df4f8" (UID: "1952bdcb-9771-45be-85ce-085a1c9df4f8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:14:40.895708 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:40.895214    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1952bdcb-9771-45be-85ce-085a1c9df4f8-istiod-ca-cert" pod "1952bdcb-9771-45be-85ce-085a1c9df4f8" (UID: "1952bdcb-9771-45be-85ce-085a1c9df4f8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:14:40.895708 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:40.895588    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:14:40.895708 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:40.895613    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:14:40.895708 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:40.895630    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:14:40.895708 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:40.895644    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1952bdcb-9771-45be-85ce-085a1c9df4f8-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:14:40.895708 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:40.895663    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:14:40.897905 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:40.897882    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-istio-envoy" pod "1952bdcb-9771-45be-85ce-085a1c9df4f8" (UID: "1952bdcb-9771-45be-85ce-085a1c9df4f8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:14:40.898048 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:40.897989    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1952bdcb-9771-45be-85ce-085a1c9df4f8-istio-token" pod "1952bdcb-9771-45be-85ce-085a1c9df4f8" (UID: "1952bdcb-9771-45be-85ce-085a1c9df4f8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:14:40.899295 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:40.899272    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1952bdcb-9771-45be-85ce-085a1c9df4f8-istio-podinfo" pod "1952bdcb-9771-45be-85ce-085a1c9df4f8" (UID: "1952bdcb-9771-45be-85ce-085a1c9df4f8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 22:14:40.899408 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:40.899383    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1952bdcb-9771-45be-85ce-085a1c9df4f8-kube-api-access-bbjd6" pod "1952bdcb-9771-45be-85ce-085a1c9df4f8" (UID: "1952bdcb-9771-45be-85ce-085a1c9df4f8"). InnerVolumeSpecName "kube-api-access-bbjd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:14:40.997366 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:40.997339    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1952bdcb-9771-45be-85ce-085a1c9df4f8-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:14:40.997366 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:40.997367    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbjd6\" (UniqueName: \"kubernetes.io/projected/1952bdcb-9771-45be-85ce-085a1c9df4f8-kube-api-access-bbjd6\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:14:40.997543 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:40.997379    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1952bdcb-9771-45be-85ce-085a1c9df4f8-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:14:40.997543 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:40.997391    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1952bdcb-9771-45be-85ce-085a1c9df4f8-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:14:41.104985 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:41.104865    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:41.190608 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:41.189782    4460 generic.go:356] "Generic (PLEG): container finished" podID="1952bdcb-9771-45be-85ce-085a1c9df4f8" containerID="5be606e49fcbafc179b3b0af46223d6eaf056c8f0c0a59f45aa1508f00c19326" exitCode=0
Oct 31 22:14:41.190608 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:41.189878    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns" event={"ID":"1952bdcb-9771-45be-85ce-085a1c9df4f8","Type":"ContainerDied","Data":"5be606e49fcbafc179b3b0af46223d6eaf056c8f0c0a59f45aa1508f00c19326"}
Oct 31 22:14:41.190608 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:41.189952    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns" event={"ID":"1952bdcb-9771-45be-85ce-085a1c9df4f8","Type":"ContainerDied","Data":"df2c502c0c5bbc47d7ed18d9e510b490ffa776ab4e1ab4b1749d72426ed83085"}
Oct 31 22:14:41.190608 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:41.189986    4460 scope.go:122] "RemoveContainer" containerID="5be606e49fcbafc179b3b0af46223d6eaf056c8f0c0a59f45aa1508f00c19326"
Oct 31 22:14:41.190608 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:41.190042    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"
Oct 31 22:14:41.207212 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:41.207191    4460 scope.go:122] "RemoveContainer" containerID="9228743d8b134ad749810fcc221f0cd706c40b43977e9644126911a28ba76242"
Oct 31 22:14:41.217521 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:41.217502    4460 scope.go:122] "RemoveContainer" containerID="5be606e49fcbafc179b3b0af46223d6eaf056c8f0c0a59f45aa1508f00c19326"
Oct 31 22:14:41.217798 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:14:41.217771    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5be606e49fcbafc179b3b0af46223d6eaf056c8f0c0a59f45aa1508f00c19326\": container with ID starting with 5be606e49fcbafc179b3b0af46223d6eaf056c8f0c0a59f45aa1508f00c19326 not found: ID does not exist" containerID="5be606e49fcbafc179b3b0af46223d6eaf056c8f0c0a59f45aa1508f00c19326"
Oct 31 22:14:41.217904 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:41.217809    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5be606e49fcbafc179b3b0af46223d6eaf056c8f0c0a59f45aa1508f00c19326"} err="failed to get container status \"5be606e49fcbafc179b3b0af46223d6eaf056c8f0c0a59f45aa1508f00c19326\": rpc error: code = NotFound desc = could not find container \"5be606e49fcbafc179b3b0af46223d6eaf056c8f0c0a59f45aa1508f00c19326\": container with ID starting with 5be606e49fcbafc179b3b0af46223d6eaf056c8f0c0a59f45aa1508f00c19326 not found: ID does not exist"
Oct 31 22:14:41.217904 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:41.217830    4460 scope.go:122] "RemoveContainer" containerID="9228743d8b134ad749810fcc221f0cd706c40b43977e9644126911a28ba76242"
Oct 31 22:14:41.218128 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:14:41.218105    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9228743d8b134ad749810fcc221f0cd706c40b43977e9644126911a28ba76242\": container with ID starting with 9228743d8b134ad749810fcc221f0cd706c40b43977e9644126911a28ba76242 not found: ID does not exist" containerID="9228743d8b134ad749810fcc221f0cd706c40b43977e9644126911a28ba76242"
Oct 31 22:14:41.218211 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:41.218134    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9228743d8b134ad749810fcc221f0cd706c40b43977e9644126911a28ba76242"} err="failed to get container status \"9228743d8b134ad749810fcc221f0cd706c40b43977e9644126911a28ba76242\": rpc error: code = NotFound desc = could not find container \"9228743d8b134ad749810fcc221f0cd706c40b43977e9644126911a28ba76242\": container with ID starting with 9228743d8b134ad749810fcc221f0cd706c40b43977e9644126911a28ba76242 not found: ID does not exist"
Oct 31 22:14:41.249847 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:41.249820    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"]
Oct 31 22:14:41.256624 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:41.256603    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-4275b9-openshift-default-67d67b45c-dbfns"]
Oct 31 22:14:42.225881 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:42.225846    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1952bdcb-9771-45be-85ce-085a1c9df4f8" path="/var/lib/kubelet/pods/1952bdcb-9771-45be-85ce-085a1c9df4f8/volumes"
Oct 31 22:14:42.841361 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:42.841326    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:43.716580 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:43.716531    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:44.669532 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:44.669489    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:46.543396 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:46.543358    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:46.782667 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:46.782632    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-a5affe-78c59c567-c6hdt"]
Oct 31 22:14:46.889241 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:46.889118    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:47.009961 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:47.009816    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:48.310551 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:48.310514    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:50.044790 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:50.044751    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:50.888056 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:50.888015    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:51.453492 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:14:51.453456    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:14:51.783796 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:51.783741    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a5affe-78c59c567-c6hdt" podUID="8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1" containerName="payload-pre-processing" containerID="cri-o://4574ea54ca431d88627d1a74c901ebbed08beafcd9a87dc16f9176aa627dd7f7" gracePeriod=25
Oct 31 22:14:51.806021 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:51.805985    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:53.517256 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:53.517214    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:55.241782 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:55.241743    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:57.038911 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:57.038873    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:58.715358 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:58.715320    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:14:59.847507 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:14:59.847469    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a5affe-78c59c567-c6hdt" podUID="8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:15:00.166875 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.166769    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd"]
Oct 31 22:15:00.169150 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.168713    4460 cpu_manager.go:414] "containerMap: removing container" podUID="1952bdcb-9771-45be-85ce-085a1c9df4f8" containerName="istio-proxy"
Oct 31 22:15:00.169150 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.168741    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1952bdcb-9771-45be-85ce-085a1c9df4f8" containerName="istio-proxy"
Oct 31 22:15:00.169150 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.169006    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="1952bdcb-9771-45be-85ce-085a1c9df4f8" containerName="istio-proxy"
Oct 31 22:15:00.175160 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.175132    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr"]
Oct 31 22:15:00.176450 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.176423    4460 cpu_manager.go:414] "containerMap: removing container" podUID="1952bdcb-9771-45be-85ce-085a1c9df4f8" containerName="istio-proxy"
Oct 31 22:15:00.176604 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.176452    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1952bdcb-9771-45be-85ce-085a1c9df4f8" containerName="istio-proxy"
Oct 31 22:15:00.176745 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.176732    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="1952bdcb-9771-45be-85ce-085a1c9df4f8" containerName="istio-proxy"
Oct 31 22:15:00.178183 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.178145    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd"
Oct 31 22:15:00.180478 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.180441    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc"]
Oct 31 22:15:00.180598 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.180582    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr"
Oct 31 22:15:00.185139 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.185114    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc"
Oct 31 22:15:00.203421 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.203381    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd"]
Oct 31 22:15:00.210119 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.210044    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh"]
Oct 31 22:15:00.214574 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.214552    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh"
Oct 31 22:15:00.270424 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.270079    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9hwvp\""
Oct 31 22:15:00.295457 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.295426    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5"]
Oct 31 22:15:00.306092 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.306071    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc"]
Oct 31 22:15:00.306251 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.306241    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5"]
Oct 31 22:15:00.308009 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.307608    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5"
Oct 31 22:15:00.311150 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.311130    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4"]
Oct 31 22:15:00.311341 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.311228    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5"
Oct 31 22:15:00.323899 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.319624    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh"]
Oct 31 22:15:00.323899 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.319685    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt"]
Oct 31 22:15:00.323899 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.319726    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4"
Oct 31 22:15:00.324261 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.324244    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr"]
Oct 31 22:15:00.324366 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.324345    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt"
Oct 31 22:15:00.324453 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.324443    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5"]
Oct 31 22:15:00.324554 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.324546    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2"]
Oct 31 22:15:00.329009 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.328994    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5"]
Oct 31 22:15:00.329133 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.329065    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2"
Oct 31 22:15:00.331810 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.331788    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29891415-tbdkz"]
Oct 31 22:15:00.338591 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.338567    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4"]
Oct 31 22:15:00.338698 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.338680    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29891415-tbdkz"
Oct 31 22:15:00.339518 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.339500    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt"]
Oct 31 22:15:00.350261 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.350236    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b29dc108-b17e-44be-9f2b-1d133f0acfe7-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc\" (UID: \"b29dc108-b17e-44be-9f2b-1d133f0acfe7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc"
Oct 31 22:15:00.350355 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.350302    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb499\" (UniqueName: \"kubernetes.io/projected/dfc291e0-afb6-4fbd-8314-2e2a36096128-kube-api-access-cb499\") pod \"maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd\" (UID: \"dfc291e0-afb6-4fbd-8314-2e2a36096128\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd"
Oct 31 22:15:00.350401 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.350376    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9z5q\" (UniqueName: \"kubernetes.io/projected/cde3b26f-a0a0-4c0f-835c-cdf613428100-kube-api-access-s9z5q\") pod \"maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh\" (UID: \"cde3b26f-a0a0-4c0f-835c-cdf613428100\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh"
Oct 31 22:15:00.350438 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.350421    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cde3b26f-a0a0-4c0f-835c-cdf613428100-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh\" (UID: \"cde3b26f-a0a0-4c0f-835c-cdf613428100\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh"
Oct 31 22:15:00.350480 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.350468    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v48h\" (UniqueName: \"kubernetes.io/projected/b29dc108-b17e-44be-9f2b-1d133f0acfe7-kube-api-access-6v48h\") pod \"maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc\" (UID: \"b29dc108-b17e-44be-9f2b-1d133f0acfe7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc"
Oct 31 22:15:00.350616 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.350599    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2f44b0f-767e-474e-8f9d-9a84dd2cf281-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr\" (UID: \"e2f44b0f-767e-474e-8f9d-9a84dd2cf281\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr"
Oct 31 22:15:00.350654 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.350630    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tktq\" (UniqueName: \"kubernetes.io/projected/e2f44b0f-767e-474e-8f9d-9a84dd2cf281-kube-api-access-6tktq\") pod \"maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr\" (UID: \"e2f44b0f-767e-474e-8f9d-9a84dd2cf281\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr"
Oct 31 22:15:00.350697 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.350683    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfc291e0-afb6-4fbd-8314-2e2a36096128-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd\" (UID: \"dfc291e0-afb6-4fbd-8314-2e2a36096128\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd"
Oct 31 22:15:00.351495 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.351472    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2"]
Oct 31 22:15:00.357014 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.356900    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29891415-tbdkz"]
Oct 31 22:15:00.453095 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.452992    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb499\" (UniqueName: \"kubernetes.io/projected/dfc291e0-afb6-4fbd-8314-2e2a36096128-kube-api-access-cb499\") pod \"maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd\" (UID: \"dfc291e0-afb6-4fbd-8314-2e2a36096128\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd"
Oct 31 22:15:00.453322 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.453088    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9z5q\" (UniqueName: \"kubernetes.io/projected/cde3b26f-a0a0-4c0f-835c-cdf613428100-kube-api-access-s9z5q\") pod \"maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh\" (UID: \"cde3b26f-a0a0-4c0f-835c-cdf613428100\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh"
Oct 31 22:15:00.453322 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.453134    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cde3b26f-a0a0-4c0f-835c-cdf613428100-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh\" (UID: \"cde3b26f-a0a0-4c0f-835c-cdf613428100\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh"
Oct 31 22:15:00.453322 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.453166    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc0b50fb-08dd-4729-bf9f-01e891bf8ca6-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4\" (UID: \"dc0b50fb-08dd-4729-bf9f-01e891bf8ca6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4"
Oct 31 22:15:00.453322 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.453200    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bbd45fa-4d2b-422b-8471-8da4428c1a6f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt\" (UID: \"9bbd45fa-4d2b-422b-8471-8da4428c1a6f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt"
Oct 31 22:15:00.453322 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.453243    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v48h\" (UniqueName: \"kubernetes.io/projected/b29dc108-b17e-44be-9f2b-1d133f0acfe7-kube-api-access-6v48h\") pod \"maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc\" (UID: \"b29dc108-b17e-44be-9f2b-1d133f0acfe7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc"
Oct 31 22:15:00.453322 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.453303    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx9ht\" (UniqueName: \"kubernetes.io/projected/c6686f63-0d95-4d42-8fcb-861e719277ce-kube-api-access-xx9ht\") pod \"maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5\" (UID: \"c6686f63-0d95-4d42-8fcb-861e719277ce\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5"
Oct 31 22:15:00.453831 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.453406    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2f44b0f-767e-474e-8f9d-9a84dd2cf281-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr\" (UID: \"e2f44b0f-767e-474e-8f9d-9a84dd2cf281\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr"
Oct 31 22:15:00.453831 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.453432    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tktq\" (UniqueName: \"kubernetes.io/projected/e2f44b0f-767e-474e-8f9d-9a84dd2cf281-kube-api-access-6tktq\") pod \"maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr\" (UID: \"e2f44b0f-767e-474e-8f9d-9a84dd2cf281\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr"
Oct 31 22:15:00.453831 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.453473    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d68f00ec-71ab-48be-b3bd-d988eeeb5a88-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2\" (UID: \"d68f00ec-71ab-48be-b3bd-d988eeeb5a88\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2"
Oct 31 22:15:00.453831 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.453516    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfc291e0-afb6-4fbd-8314-2e2a36096128-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd\" (UID: \"dfc291e0-afb6-4fbd-8314-2e2a36096128\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd"
Oct 31 22:15:00.453831 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.453557    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txbmg\" (UniqueName: \"kubernetes.io/projected/ec53414f-1bc8-4065-819c-d85620195d70-kube-api-access-txbmg\") pod \"maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5\" (UID: \"ec53414f-1bc8-4065-819c-d85620195d70\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5"
Oct 31 22:15:00.453831 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.453589    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x674f\" (UniqueName: \"kubernetes.io/projected/d68f00ec-71ab-48be-b3bd-d988eeeb5a88-kube-api-access-x674f\") pod \"maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2\" (UID: \"d68f00ec-71ab-48be-b3bd-d988eeeb5a88\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2"
Oct 31 22:15:00.453831 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.453631    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6686f63-0d95-4d42-8fcb-861e719277ce-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5\" (UID: \"c6686f63-0d95-4d42-8fcb-861e719277ce\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5"
Oct 31 22:15:00.453831 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.453719    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdtnt\" (UniqueName: \"kubernetes.io/projected/dc0b50fb-08dd-4729-bf9f-01e891bf8ca6-kube-api-access-xdtnt\") pod \"maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4\" (UID: \"dc0b50fb-08dd-4729-bf9f-01e891bf8ca6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4"
Oct 31 22:15:00.453831 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.453755    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42f6904e-6141-41a7-a7c5-ee9c766193d2-maas-api-service-ca\") pod \"maas-api-key-cleanup-29891415-tbdkz\" (UID: \"42f6904e-6141-41a7-a7c5-ee9c766193d2\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29891415-tbdkz"
Oct 31 22:15:00.453831 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.453806    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsc6q\" (UniqueName: \"kubernetes.io/projected/42f6904e-6141-41a7-a7c5-ee9c766193d2-kube-api-access-fsc6q\") pod \"maas-api-key-cleanup-29891415-tbdkz\" (UID: \"42f6904e-6141-41a7-a7c5-ee9c766193d2\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29891415-tbdkz"
Oct 31 22:15:00.454419 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.453903    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwrbk\" (UniqueName: \"kubernetes.io/projected/9bbd45fa-4d2b-422b-8471-8da4428c1a6f-kube-api-access-fwrbk\") pod \"maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt\" (UID: \"9bbd45fa-4d2b-422b-8471-8da4428c1a6f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt"
Oct 31 22:15:00.454419 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.453934    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec53414f-1bc8-4065-819c-d85620195d70-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5\" (UID: \"ec53414f-1bc8-4065-819c-d85620195d70\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5"
Oct 31 22:15:00.454419 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.454005    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b29dc108-b17e-44be-9f2b-1d133f0acfe7-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc\" (UID: \"b29dc108-b17e-44be-9f2b-1d133f0acfe7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc"
Oct 31 22:15:00.454867 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.454837    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b29dc108-b17e-44be-9f2b-1d133f0acfe7-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc\" (UID: \"b29dc108-b17e-44be-9f2b-1d133f0acfe7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc"
Oct 31 22:15:00.455233 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.455198    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfc291e0-afb6-4fbd-8314-2e2a36096128-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd\" (UID: \"dfc291e0-afb6-4fbd-8314-2e2a36096128\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd"
Oct 31 22:15:00.455856 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.455833    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cde3b26f-a0a0-4c0f-835c-cdf613428100-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh\" (UID: \"cde3b26f-a0a0-4c0f-835c-cdf613428100\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh"
Oct 31 22:15:00.456010 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.455877    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2f44b0f-767e-474e-8f9d-9a84dd2cf281-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr\" (UID: \"e2f44b0f-767e-474e-8f9d-9a84dd2cf281\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr"
Oct 31 22:15:00.473388 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.473358    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:00.481744 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.481715    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9z5q\" (UniqueName: \"kubernetes.io/projected/cde3b26f-a0a0-4c0f-835c-cdf613428100-kube-api-access-s9z5q\") pod \"maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh\" (UID: \"cde3b26f-a0a0-4c0f-835c-cdf613428100\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh"
Oct 31 22:15:00.482499 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.482465    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tktq\" (UniqueName: \"kubernetes.io/projected/e2f44b0f-767e-474e-8f9d-9a84dd2cf281-kube-api-access-6tktq\") pod \"maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr\" (UID: \"e2f44b0f-767e-474e-8f9d-9a84dd2cf281\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr"
Oct 31 22:15:00.482696 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.482670    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v48h\" (UniqueName: \"kubernetes.io/projected/b29dc108-b17e-44be-9f2b-1d133f0acfe7-kube-api-access-6v48h\") pod \"maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc\" (UID: \"b29dc108-b17e-44be-9f2b-1d133f0acfe7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc"
Oct 31 22:15:00.482696 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.482691    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb499\" (UniqueName: \"kubernetes.io/projected/dfc291e0-afb6-4fbd-8314-2e2a36096128-kube-api-access-cb499\") pod \"maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd\" (UID: \"dfc291e0-afb6-4fbd-8314-2e2a36096128\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd"
Oct 31 22:15:00.570966 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.570678    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwrbk\" (UniqueName: \"kubernetes.io/projected/9bbd45fa-4d2b-422b-8471-8da4428c1a6f-kube-api-access-fwrbk\") pod \"maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt\" (UID: \"9bbd45fa-4d2b-422b-8471-8da4428c1a6f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt"
Oct 31 22:15:00.570966 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.570753    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec53414f-1bc8-4065-819c-d85620195d70-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5\" (UID: \"ec53414f-1bc8-4065-819c-d85620195d70\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5"
Oct 31 22:15:00.570966 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.570901    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc0b50fb-08dd-4729-bf9f-01e891bf8ca6-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4\" (UID: \"dc0b50fb-08dd-4729-bf9f-01e891bf8ca6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4"
Oct 31 22:15:00.570966 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.570932    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bbd45fa-4d2b-422b-8471-8da4428c1a6f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt\" (UID: \"9bbd45fa-4d2b-422b-8471-8da4428c1a6f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt"
Oct 31 22:15:00.571381 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.571042    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx9ht\" (UniqueName: \"kubernetes.io/projected/c6686f63-0d95-4d42-8fcb-861e719277ce-kube-api-access-xx9ht\") pod \"maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5\" (UID: \"c6686f63-0d95-4d42-8fcb-861e719277ce\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5"
Oct 31 22:15:00.571381 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.571168    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d68f00ec-71ab-48be-b3bd-d988eeeb5a88-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2\" (UID: \"d68f00ec-71ab-48be-b3bd-d988eeeb5a88\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2"
Oct 31 22:15:00.571381 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.571238    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txbmg\" (UniqueName: \"kubernetes.io/projected/ec53414f-1bc8-4065-819c-d85620195d70-kube-api-access-txbmg\") pod \"maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5\" (UID: \"ec53414f-1bc8-4065-819c-d85620195d70\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5"
Oct 31 22:15:00.571381 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.571284    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x674f\" (UniqueName: \"kubernetes.io/projected/d68f00ec-71ab-48be-b3bd-d988eeeb5a88-kube-api-access-x674f\") pod \"maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2\" (UID: \"d68f00ec-71ab-48be-b3bd-d988eeeb5a88\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2"
Oct 31 22:15:00.571381 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.571324    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6686f63-0d95-4d42-8fcb-861e719277ce-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5\" (UID: \"c6686f63-0d95-4d42-8fcb-861e719277ce\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5"
Oct 31 22:15:00.571663 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.571432    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdtnt\" (UniqueName: \"kubernetes.io/projected/dc0b50fb-08dd-4729-bf9f-01e891bf8ca6-kube-api-access-xdtnt\") pod \"maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4\" (UID: \"dc0b50fb-08dd-4729-bf9f-01e891bf8ca6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4"
Oct 31 22:15:00.571663 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.571467    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42f6904e-6141-41a7-a7c5-ee9c766193d2-maas-api-service-ca\") pod \"maas-api-key-cleanup-29891415-tbdkz\" (UID: \"42f6904e-6141-41a7-a7c5-ee9c766193d2\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29891415-tbdkz"
Oct 31 22:15:00.571663 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.571531    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsc6q\" (UniqueName: \"kubernetes.io/projected/42f6904e-6141-41a7-a7c5-ee9c766193d2-kube-api-access-fsc6q\") pod \"maas-api-key-cleanup-29891415-tbdkz\" (UID: \"42f6904e-6141-41a7-a7c5-ee9c766193d2\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29891415-tbdkz"
Oct 31 22:15:00.582980 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.581120    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc0b50fb-08dd-4729-bf9f-01e891bf8ca6-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4\" (UID: \"dc0b50fb-08dd-4729-bf9f-01e891bf8ca6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4"
Oct 31 22:15:00.582980 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.581352    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6686f63-0d95-4d42-8fcb-861e719277ce-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5\" (UID: \"c6686f63-0d95-4d42-8fcb-861e719277ce\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5"
Oct 31 22:15:00.582980 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.581819    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bbd45fa-4d2b-422b-8471-8da4428c1a6f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt\" (UID: \"9bbd45fa-4d2b-422b-8471-8da4428c1a6f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt"
Oct 31 22:15:00.582980 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.582708    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d68f00ec-71ab-48be-b3bd-d988eeeb5a88-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2\" (UID: \"d68f00ec-71ab-48be-b3bd-d988eeeb5a88\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2"
Oct 31 22:15:00.582980 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.582920    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42f6904e-6141-41a7-a7c5-ee9c766193d2-maas-api-service-ca\") pod \"maas-api-key-cleanup-29891415-tbdkz\" (UID: \"42f6904e-6141-41a7-a7c5-ee9c766193d2\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29891415-tbdkz"
Oct 31 22:15:00.599966 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.593019    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec53414f-1bc8-4065-819c-d85620195d70-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5\" (UID: \"ec53414f-1bc8-4065-819c-d85620195d70\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5"
Oct 31 22:15:00.618983 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.602748    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9hwvp\""
Oct 31 22:15:00.618983 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.605886    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr"
Oct 31 22:15:00.618983 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.610775    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc"
Oct 31 22:15:00.618983 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.608564    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd"
Oct 31 22:15:00.618983 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.608392    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:00.618983 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.616295    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh"
Oct 31 22:15:00.690581 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.686656    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwrbk\" (UniqueName: \"kubernetes.io/projected/9bbd45fa-4d2b-422b-8471-8da4428c1a6f-kube-api-access-fwrbk\") pod \"maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt\" (UID: \"9bbd45fa-4d2b-422b-8471-8da4428c1a6f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt"
Oct 31 22:15:00.690581 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.686864    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x674f\" (UniqueName: \"kubernetes.io/projected/d68f00ec-71ab-48be-b3bd-d988eeeb5a88-kube-api-access-x674f\") pod \"maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2\" (UID: \"d68f00ec-71ab-48be-b3bd-d988eeeb5a88\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2"
Oct 31 22:15:00.690581 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.687524    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx9ht\" (UniqueName: \"kubernetes.io/projected/c6686f63-0d95-4d42-8fcb-861e719277ce-kube-api-access-xx9ht\") pod \"maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5\" (UID: \"c6686f63-0d95-4d42-8fcb-861e719277ce\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5"
Oct 31 22:15:00.690581 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.688164    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdtnt\" (UniqueName: \"kubernetes.io/projected/dc0b50fb-08dd-4729-bf9f-01e891bf8ca6-kube-api-access-xdtnt\") pod \"maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4\" (UID: \"dc0b50fb-08dd-4729-bf9f-01e891bf8ca6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4"
Oct 31 22:15:00.690581 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.690349    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2"
Oct 31 22:15:00.696071 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.692465    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txbmg\" (UniqueName: \"kubernetes.io/projected/ec53414f-1bc8-4065-819c-d85620195d70-kube-api-access-txbmg\") pod \"maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5\" (UID: \"ec53414f-1bc8-4065-819c-d85620195d70\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5"
Oct 31 22:15:00.705778 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.704119    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsc6q\" (UniqueName: \"kubernetes.io/projected/42f6904e-6141-41a7-a7c5-ee9c766193d2-kube-api-access-fsc6q\") pod \"maas-api-key-cleanup-29891415-tbdkz\" (UID: \"42f6904e-6141-41a7-a7c5-ee9c766193d2\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29891415-tbdkz"
Oct 31 22:15:00.929076 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.929048    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5"
Oct 31 22:15:00.942908 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.942881    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5"
Oct 31 22:15:00.958310 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.958245    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4"
Oct 31 22:15:00.961375 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.961349    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cd6e81c8-th99j"]
Oct 31 22:15:00.966727 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.966701    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd6e81c8-th99j"
Oct 31 22:15:00.980466 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.980267    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt"
Oct 31 22:15:00.991018 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.984758    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 22:15:00.995858 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.993883    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cd6e81c8-th99j"]
Oct 31 22:15:00.997726 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:00.997544    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29891415-tbdkz"
Oct 31 22:15:01.055999 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:01.054966    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr"]
Oct 31 22:15:01.067718 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:01.067689    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc"]
Oct 31 22:15:01.131397 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:01.130866    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fl6r4\" (UniqueName: \"kubernetes.io/projected/4b94e7af-e6fa-434b-b1ac-3069e1d6953e-kube-api-access-fl6r4\") pod \"maas-api-revoke-sub-cd6e81c8-th99j\" (UID: \"4b94e7af-e6fa-434b-b1ac-3069e1d6953e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd6e81c8-th99j"
Oct 31 22:15:01.131397 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:01.131016    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4b94e7af-e6fa-434b-b1ac-3069e1d6953e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cd6e81c8-th99j\" (UID: \"4b94e7af-e6fa-434b-b1ac-3069e1d6953e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd6e81c8-th99j"
Oct 31 22:15:01.131397 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:01.131216    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4b94e7af-e6fa-434b-b1ac-3069e1d6953e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cd6e81c8-th99j\" (UID: \"4b94e7af-e6fa-434b-b1ac-3069e1d6953e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd6e81c8-th99j"
Oct 31 22:15:01.234532 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:01.233503    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fl6r4\" (UniqueName: \"kubernetes.io/projected/4b94e7af-e6fa-434b-b1ac-3069e1d6953e-kube-api-access-fl6r4\") pod \"maas-api-revoke-sub-cd6e81c8-th99j\" (UID: \"4b94e7af-e6fa-434b-b1ac-3069e1d6953e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd6e81c8-th99j"
Oct 31 22:15:01.234532 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:01.233607    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4b94e7af-e6fa-434b-b1ac-3069e1d6953e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cd6e81c8-th99j\" (UID: \"4b94e7af-e6fa-434b-b1ac-3069e1d6953e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd6e81c8-th99j"
Oct 31 22:15:01.234532 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:01.233829    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4b94e7af-e6fa-434b-b1ac-3069e1d6953e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cd6e81c8-th99j\" (UID: \"4b94e7af-e6fa-434b-b1ac-3069e1d6953e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd6e81c8-th99j"
Oct 31 22:15:01.239637 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:01.239539    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4b94e7af-e6fa-434b-b1ac-3069e1d6953e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cd6e81c8-th99j\" (UID: \"4b94e7af-e6fa-434b-b1ac-3069e1d6953e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd6e81c8-th99j"
Oct 31 22:15:01.265018 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:01.264932    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2"]
Oct 31 22:15:01.276432 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:01.271552    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh"]
Oct 31 22:15:01.284657 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:01.281961    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd"]
Oct 31 22:15:01.285993 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:01.285890    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5"]
Oct 31 22:15:01.330753 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:01.327657    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5"]
Oct 31 22:15:01.345417 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:01.338595    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fl6r4\" (UniqueName: \"kubernetes.io/projected/4b94e7af-e6fa-434b-b1ac-3069e1d6953e-kube-api-access-fl6r4\") pod \"maas-api-revoke-sub-cd6e81c8-th99j\" (UID: \"4b94e7af-e6fa-434b-b1ac-3069e1d6953e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd6e81c8-th99j"
Oct 31 22:15:01.345417 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:01.341494    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4b94e7af-e6fa-434b-b1ac-3069e1d6953e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cd6e81c8-th99j\" (UID: \"4b94e7af-e6fa-434b-b1ac-3069e1d6953e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd6e81c8-th99j"
Oct 31 22:15:01.396286 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:01.396251    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5" event={"ID":"c6686f63-0d95-4d42-8fcb-861e719277ce","Type":"ContainerStarted","Data":"a2bdb638c941b977933b19d45054e2bbaa02bbcb5eb3e8a030ed4097ca85d0b5"}
Oct 31 22:15:01.405391 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:01.405367    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt"]
Oct 31 22:15:01.408884 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:01.408856    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2" event={"ID":"d68f00ec-71ab-48be-b3bd-d988eeeb5a88","Type":"ContainerStarted","Data":"b1fc41f803566e69bd2af141fbf517faa99af0ea122933f5ad5e443851b063e6"}
Oct 31 22:15:01.410883 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:01.410541    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc" event={"ID":"b29dc108-b17e-44be-9f2b-1d133f0acfe7","Type":"ContainerStarted","Data":"24c9f12df86a7f4bfec4f5f8fde73a4cd57ce93dff68205cf2a57036be691314"}
Oct 31 22:15:01.411169 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:01.410959    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4"]
Oct 31 22:15:01.413141 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:01.412320    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5" event={"ID":"ec53414f-1bc8-4065-819c-d85620195d70","Type":"ContainerStarted","Data":"311eb5916d8b489116bc192ae79c05cd5c78ac81597f6d114f841ce3cccaaaa5"}
Oct 31 22:15:01.414186 ip-10-0-136-157 kubenswrapper[4460]: W1031 22:15:01.414156    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc0b50fb_08dd_4729_bf9f_01e891bf8ca6.slice/crio-d0d79de56d174c263b1cb9dd5ae08b85ab979ba38f2f794de0e07f33e4e0916b WatchSource:0}: Error finding container d0d79de56d174c263b1cb9dd5ae08b85ab979ba38f2f794de0e07f33e4e0916b: Status 404 returned error can't find the container with id d0d79de56d174c263b1cb9dd5ae08b85ab979ba38f2f794de0e07f33e4e0916b
Oct 31 22:15:01.416779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:01.416748    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh" event={"ID":"cde3b26f-a0a0-4c0f-835c-cdf613428100","Type":"ContainerStarted","Data":"f7f85d75252f1763575001e9ff71469ac5d6949853aff89fff07b8ee5139ff60"}
Oct 31 22:15:01.421369 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:01.421169    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd" event={"ID":"dfc291e0-afb6-4fbd-8314-2e2a36096128","Type":"ContainerStarted","Data":"fd8c953277fcdb7e150b2a9bff4e7898ca533c1b4c586c035651b7455df23ecd"}
Oct 31 22:15:01.434971 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:01.434517    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29891415-tbdkz"]
Oct 31 22:15:01.438859 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:01.438488    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr" event={"ID":"e2f44b0f-767e-474e-8f9d-9a84dd2cf281","Type":"ContainerStarted","Data":"85853de135d1a09ab98c56f6dfc8bf1aef11118e42bd18625488573649c439da"}
Oct 31 22:15:01.526631 ip-10-0-136-157 kubenswrapper[4460]: W1031 22:15:01.526595    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42f6904e_6141_41a7_a7c5_ee9c766193d2.slice/crio-3a121fd3c0cbffc66819a72b718dc11a43af9aa9fdeb7e9c9029588fe1b1fbc9 WatchSource:0}: Error finding container 3a121fd3c0cbffc66819a72b718dc11a43af9aa9fdeb7e9c9029588fe1b1fbc9: Status 404 returned error can't find the container with id 3a121fd3c0cbffc66819a72b718dc11a43af9aa9fdeb7e9c9029588fe1b1fbc9
Oct 31 22:15:01.606139 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:01.606109    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd6e81c8-th99j"
Oct 31 22:15:02.015587 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:02.015540    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cd6e81c8-th99j"]
Oct 31 22:15:02.029458 ip-10-0-136-157 kubenswrapper[4460]: W1031 22:15:02.029416    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b94e7af_e6fa_434b_b1ac_3069e1d6953e.slice/crio-ba5a31b52c3ce61aed13a3bc560d8d0613016ea28624e1ef44962716b21303b1 WatchSource:0}: Error finding container ba5a31b52c3ce61aed13a3bc560d8d0613016ea28624e1ef44962716b21303b1: Status 404 returned error can't find the container with id ba5a31b52c3ce61aed13a3bc560d8d0613016ea28624e1ef44962716b21303b1
Oct 31 22:15:02.374915 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:02.374888    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:02.492993 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:02.492914    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd6e81c8-th99j" event={"ID":"4b94e7af-e6fa-434b-b1ac-3069e1d6953e","Type":"ContainerStarted","Data":"ba5a31b52c3ce61aed13a3bc560d8d0613016ea28624e1ef44962716b21303b1"}
Oct 31 22:15:02.495052 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:02.495009    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh" event={"ID":"cde3b26f-a0a0-4c0f-835c-cdf613428100","Type":"ContainerStarted","Data":"42755992be5a9873f028f90098550fb3ea0c361b19cdcb5adce5637b0e14b05c"}
Oct 31 22:15:02.497004 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:02.496973    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd" event={"ID":"dfc291e0-afb6-4fbd-8314-2e2a36096128","Type":"ContainerStarted","Data":"859e425ac014d253b709645b20dbffc3d952b9fc5996a801dae95bfb485cf94c"}
Oct 31 22:15:02.498686 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:02.498662    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr" event={"ID":"e2f44b0f-767e-474e-8f9d-9a84dd2cf281","Type":"ContainerStarted","Data":"f5355e523bed9a473601c57987789b58578907b1dfed629d277828f9e3629c25"}
Oct 31 22:15:02.498887 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:02.498868    4460 scope.go:122] "RemoveContainer" containerID="f5355e523bed9a473601c57987789b58578907b1dfed629d277828f9e3629c25"
Oct 31 22:15:02.500394 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:02.500370    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt" event={"ID":"9bbd45fa-4d2b-422b-8471-8da4428c1a6f","Type":"ContainerStarted","Data":"486f373479888e14cbac2436cab39d9d77350e59808390311b830837a0c218fe"}
Oct 31 22:15:02.502338 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:02.502315    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5" event={"ID":"c6686f63-0d95-4d42-8fcb-861e719277ce","Type":"ContainerStarted","Data":"750468a3c57f9ae7d50199b063ef98764c9bfce42b5b057436959f86f4ab19eb"}
Oct 31 22:15:02.505880 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:02.505857    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2" event={"ID":"d68f00ec-71ab-48be-b3bd-d988eeeb5a88","Type":"ContainerStarted","Data":"ee6bd2a8503af4332cd2b190e7c7e78c63f3b13b7eeb772714666fbd3d545332"}
Oct 31 22:15:02.507486 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:02.507462    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4" event={"ID":"dc0b50fb-08dd-4729-bf9f-01e891bf8ca6","Type":"ContainerStarted","Data":"b0f5734d1211594df55db9c64304547f794efe73d1f6ae6d7e326ef235a63b27"}
Oct 31 22:15:02.507592 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:02.507506    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4" event={"ID":"dc0b50fb-08dd-4729-bf9f-01e891bf8ca6","Type":"ContainerStarted","Data":"d0d79de56d174c263b1cb9dd5ae08b85ab979ba38f2f794de0e07f33e4e0916b"}
Oct 31 22:15:02.509218 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:02.509198    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc" event={"ID":"b29dc108-b17e-44be-9f2b-1d133f0acfe7","Type":"ContainerStarted","Data":"cc56b700ee1366a4d0ce8d4afc2447edf9542948d54eaa5e90f162b2bf47315f"}
Oct 31 22:15:02.509426 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:02.509392    4460 scope.go:122] "RemoveContainer" containerID="cc56b700ee1366a4d0ce8d4afc2447edf9542948d54eaa5e90f162b2bf47315f"
Oct 31 22:15:02.510960 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:02.510917    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29891415-tbdkz" event={"ID":"42f6904e-6141-41a7-a7c5-ee9c766193d2","Type":"ContainerStarted","Data":"3a121fd3c0cbffc66819a72b718dc11a43af9aa9fdeb7e9c9029588fe1b1fbc9"}
Oct 31 22:15:02.512900 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:02.512872    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5" event={"ID":"ec53414f-1bc8-4065-819c-d85620195d70","Type":"ContainerStarted","Data":"e7e0098577c224bf355aab3f415a5b22d34952b9a0c1108ea44835048acf3751"}
Oct 31 22:15:02.541899 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:02.541842    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh" podStartSLOduration=2.5418281289999998 podStartE2EDuration="2.541828129s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:15:02.538634956 +0000 UTC m=+3542.454944403" watchObservedRunningTime="2026-10-31 22:15:02.541828129 +0000 UTC m=+3542.458137583"
Oct 31 22:15:02.587759 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:02.587703    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5" podStartSLOduration=2.587677067 podStartE2EDuration="2.587677067s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:15:02.584441143 +0000 UTC m=+3542.500750573" watchObservedRunningTime="2026-10-31 22:15:02.587677067 +0000 UTC m=+3542.503986498"
Oct 31 22:15:02.632370 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:02.632317    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4" podStartSLOduration=2.6323044920000003 podStartE2EDuration="2.632304492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:15:02.627312721 +0000 UTC m=+3542.543622148" watchObservedRunningTime="2026-10-31 22:15:02.632304492 +0000 UTC m=+3542.548613921"
Oct 31 22:15:02.719595 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:02.719483    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5" podStartSLOduration=2.719468857 podStartE2EDuration="2.719468857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:15:02.715641945 +0000 UTC m=+3542.631951378" watchObservedRunningTime="2026-10-31 22:15:02.719468857 +0000 UTC m=+3542.635778286"
Oct 31 22:15:02.759004 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:02.758953    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2" podStartSLOduration=2.758923701 podStartE2EDuration="2.758923701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:15:02.758352112 +0000 UTC m=+3542.674661541" watchObservedRunningTime="2026-10-31 22:15:02.758923701 +0000 UTC m=+3542.675233130"
Oct 31 22:15:02.802602 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:02.802555    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29891415-tbdkz" podStartSLOduration=2.802541453 podStartE2EDuration="2.802541453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:15:02.801291379 +0000 UTC m=+3542.717600810" watchObservedRunningTime="2026-10-31 22:15:02.802541453 +0000 UTC m=+3542.718850883"
Oct 31 22:15:02.846738 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:02.846687    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd" podStartSLOduration=2.846671424 podStartE2EDuration="2.846671424s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:15:02.845282893 +0000 UTC m=+3542.761592351" watchObservedRunningTime="2026-10-31 22:15:02.846671424 +0000 UTC m=+3542.762980854"
Oct 31 22:15:03.523060 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.523026    4460 generic.go:356] "Generic (PLEG): container finished" podID="b29dc108-b17e-44be-9f2b-1d133f0acfe7" containerID="cc56b700ee1366a4d0ce8d4afc2447edf9542948d54eaa5e90f162b2bf47315f" exitCode=60
Oct 31 22:15:03.523523 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.523125    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc" event={"ID":"b29dc108-b17e-44be-9f2b-1d133f0acfe7","Type":"ContainerDied","Data":"cc56b700ee1366a4d0ce8d4afc2447edf9542948d54eaa5e90f162b2bf47315f"}
Oct 31 22:15:03.523523 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.523170    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc" event={"ID":"b29dc108-b17e-44be-9f2b-1d133f0acfe7","Type":"ContainerStarted","Data":"c3cd7b0c27daa58819ba3b85a1599f748fc1ca12d477610182d24e7834b381ab"}
Oct 31 22:15:03.525171 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.525138    4460 generic.go:356] "Generic (PLEG): container finished" podID="42f6904e-6141-41a7-a7c5-ee9c766193d2" containerID="40674a7a010de7f79eaaa26d08778461ef381f50cc08753f692042b23a2f11f2" exitCode=60
Oct 31 22:15:03.525324 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.525233    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29891415-tbdkz" event={"ID":"42f6904e-6141-41a7-a7c5-ee9c766193d2","Type":"ContainerDied","Data":"40674a7a010de7f79eaaa26d08778461ef381f50cc08753f692042b23a2f11f2"}
Oct 31 22:15:03.525595 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.525572    4460 scope.go:122] "RemoveContainer" containerID="40674a7a010de7f79eaaa26d08778461ef381f50cc08753f692042b23a2f11f2"
Oct 31 22:15:03.527082 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.527053    4460 generic.go:356] "Generic (PLEG): container finished" podID="ec53414f-1bc8-4065-819c-d85620195d70" containerID="e7e0098577c224bf355aab3f415a5b22d34952b9a0c1108ea44835048acf3751" exitCode=60
Oct 31 22:15:03.527171 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.527133    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5" event={"ID":"ec53414f-1bc8-4065-819c-d85620195d70","Type":"ContainerDied","Data":"e7e0098577c224bf355aab3f415a5b22d34952b9a0c1108ea44835048acf3751"}
Oct 31 22:15:03.527416 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.527361    4460 scope.go:122] "RemoveContainer" containerID="e7e0098577c224bf355aab3f415a5b22d34952b9a0c1108ea44835048acf3751"
Oct 31 22:15:03.529116 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.529087    4460 generic.go:356] "Generic (PLEG): container finished" podID="4b94e7af-e6fa-434b-b1ac-3069e1d6953e" containerID="897b81ab5fda8d89980120ea46da383e207a21c26d76e9796a53e35fba4ea5f6" exitCode=0
Oct 31 22:15:03.529174 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.529113    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd6e81c8-th99j" event={"ID":"4b94e7af-e6fa-434b-b1ac-3069e1d6953e","Type":"ContainerDied","Data":"897b81ab5fda8d89980120ea46da383e207a21c26d76e9796a53e35fba4ea5f6"}
Oct 31 22:15:03.532736 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.532017    4460 generic.go:356] "Generic (PLEG): container finished" podID="cde3b26f-a0a0-4c0f-835c-cdf613428100" containerID="42755992be5a9873f028f90098550fb3ea0c361b19cdcb5adce5637b0e14b05c" exitCode=60
Oct 31 22:15:03.532736 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.532047    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh" event={"ID":"cde3b26f-a0a0-4c0f-835c-cdf613428100","Type":"ContainerDied","Data":"42755992be5a9873f028f90098550fb3ea0c361b19cdcb5adce5637b0e14b05c"}
Oct 31 22:15:03.532736 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.532363    4460 scope.go:122] "RemoveContainer" containerID="42755992be5a9873f028f90098550fb3ea0c361b19cdcb5adce5637b0e14b05c"
Oct 31 22:15:03.533808 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.533784    4460 generic.go:356] "Generic (PLEG): container finished" podID="dfc291e0-afb6-4fbd-8314-2e2a36096128" containerID="859e425ac014d253b709645b20dbffc3d952b9fc5996a801dae95bfb485cf94c" exitCode=60
Oct 31 22:15:03.533926 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.533863    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd" event={"ID":"dfc291e0-afb6-4fbd-8314-2e2a36096128","Type":"ContainerDied","Data":"859e425ac014d253b709645b20dbffc3d952b9fc5996a801dae95bfb485cf94c"}
Oct 31 22:15:03.534102 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.534084    4460 scope.go:122] "RemoveContainer" containerID="859e425ac014d253b709645b20dbffc3d952b9fc5996a801dae95bfb485cf94c"
Oct 31 22:15:03.536152 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.536131    4460 generic.go:356] "Generic (PLEG): container finished" podID="e2f44b0f-767e-474e-8f9d-9a84dd2cf281" containerID="f5355e523bed9a473601c57987789b58578907b1dfed629d277828f9e3629c25" exitCode=60
Oct 31 22:15:03.536246 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.536207    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr" event={"ID":"e2f44b0f-767e-474e-8f9d-9a84dd2cf281","Type":"ContainerDied","Data":"f5355e523bed9a473601c57987789b58578907b1dfed629d277828f9e3629c25"}
Oct 31 22:15:03.536320 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.536255    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr" event={"ID":"e2f44b0f-767e-474e-8f9d-9a84dd2cf281","Type":"ContainerStarted","Data":"1aec227b232f23080f617afe207df5820f0ff495c26bb58a61b573f5624b63fe"}
Oct 31 22:15:03.538290 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.538217    4460 generic.go:356] "Generic (PLEG): container finished" podID="9bbd45fa-4d2b-422b-8471-8da4428c1a6f" containerID="4774fbe4a4e120572f332326ed471b1fe476daacafdc6c6be5473d1cb45aacb8" exitCode=60
Oct 31 22:15:03.538393 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.538290    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt" event={"ID":"9bbd45fa-4d2b-422b-8471-8da4428c1a6f","Type":"ContainerDied","Data":"4774fbe4a4e120572f332326ed471b1fe476daacafdc6c6be5473d1cb45aacb8"}
Oct 31 22:15:03.538635 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.538608    4460 scope.go:122] "RemoveContainer" containerID="4774fbe4a4e120572f332326ed471b1fe476daacafdc6c6be5473d1cb45aacb8"
Oct 31 22:15:03.540225 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.540205    4460 generic.go:356] "Generic (PLEG): container finished" podID="c6686f63-0d95-4d42-8fcb-861e719277ce" containerID="750468a3c57f9ae7d50199b063ef98764c9bfce42b5b057436959f86f4ab19eb" exitCode=60
Oct 31 22:15:03.540382 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.540363    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5" event={"ID":"c6686f63-0d95-4d42-8fcb-861e719277ce","Type":"ContainerDied","Data":"750468a3c57f9ae7d50199b063ef98764c9bfce42b5b057436959f86f4ab19eb"}
Oct 31 22:15:03.540478 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.540463    4460 scope.go:122] "RemoveContainer" containerID="750468a3c57f9ae7d50199b063ef98764c9bfce42b5b057436959f86f4ab19eb"
Oct 31 22:15:03.542406 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.542357    4460 generic.go:356] "Generic (PLEG): container finished" podID="d68f00ec-71ab-48be-b3bd-d988eeeb5a88" containerID="ee6bd2a8503af4332cd2b190e7c7e78c63f3b13b7eeb772714666fbd3d545332" exitCode=60
Oct 31 22:15:03.542489 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.542434    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2" event={"ID":"d68f00ec-71ab-48be-b3bd-d988eeeb5a88","Type":"ContainerDied","Data":"ee6bd2a8503af4332cd2b190e7c7e78c63f3b13b7eeb772714666fbd3d545332"}
Oct 31 22:15:03.542663 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.542648    4460 scope.go:122] "RemoveContainer" containerID="ee6bd2a8503af4332cd2b190e7c7e78c63f3b13b7eeb772714666fbd3d545332"
Oct 31 22:15:03.544326 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.544295    4460 generic.go:356] "Generic (PLEG): container finished" podID="dc0b50fb-08dd-4729-bf9f-01e891bf8ca6" containerID="b0f5734d1211594df55db9c64304547f794efe73d1f6ae6d7e326ef235a63b27" exitCode=60
Oct 31 22:15:03.544417 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.544333    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4" event={"ID":"dc0b50fb-08dd-4729-bf9f-01e891bf8ca6","Type":"ContainerDied","Data":"b0f5734d1211594df55db9c64304547f794efe73d1f6ae6d7e326ef235a63b27"}
Oct 31 22:15:03.544603 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.544586    4460 scope.go:122] "RemoveContainer" containerID="b0f5734d1211594df55db9c64304547f794efe73d1f6ae6d7e326ef235a63b27"
Oct 31 22:15:03.569644 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.568922    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc" podStartSLOduration=3.568902879 podStartE2EDuration="3.568902879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:15:03.566961709 +0000 UTC m=+3543.483271140" watchObservedRunningTime="2026-10-31 22:15:03.568902879 +0000 UTC m=+3543.485212309"
Oct 31 22:15:03.615903 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:03.615848    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr" podStartSLOduration=3.615828219 podStartE2EDuration="3.615828219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:15:03.614326781 +0000 UTC m=+3543.530636213" watchObservedRunningTime="2026-10-31 22:15:03.615828219 +0000 UTC m=+3543.532137650"
Oct 31 22:15:04.229506 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:04.229475    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:04.563874 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:04.563827    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29891415-tbdkz" event={"ID":"42f6904e-6141-41a7-a7c5-ee9c766193d2","Type":"ContainerStarted","Data":"8ad4ef8ff495ef5d5bb835bd62323b595cd0bf5cd1a0e3eeafc173795d146df4"}
Oct 31 22:15:04.566006 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:04.565978    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5" event={"ID":"ec53414f-1bc8-4065-819c-d85620195d70","Type":"ContainerStarted","Data":"3c97770f1e8269e433faaa85684ac97da7501543316a2d4f510a6c6f26f26902"}
Oct 31 22:15:04.568186 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:04.568156    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh" event={"ID":"cde3b26f-a0a0-4c0f-835c-cdf613428100","Type":"ContainerStarted","Data":"6bb0cc8e0300ed4c05d3a3d855fbde5a42afc05b45b73e0e5191a82757894c34"}
Oct 31 22:15:04.570259 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:04.570236    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd" event={"ID":"dfc291e0-afb6-4fbd-8314-2e2a36096128","Type":"ContainerStarted","Data":"7e3ea6d913e4e73ce104fa49a10d073547764b53e0d0dd5af578954d1590bf4d"}
Oct 31 22:15:04.572060 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:04.572034    4460 generic.go:356] "Generic (PLEG): container finished" podID="e2f44b0f-767e-474e-8f9d-9a84dd2cf281" containerID="1aec227b232f23080f617afe207df5820f0ff495c26bb58a61b573f5624b63fe" exitCode=60
Oct 31 22:15:04.572149 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:04.572108    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr" event={"ID":"e2f44b0f-767e-474e-8f9d-9a84dd2cf281","Type":"ContainerDied","Data":"1aec227b232f23080f617afe207df5820f0ff495c26bb58a61b573f5624b63fe"}
Oct 31 22:15:04.572194 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:04.572154    4460 scope.go:122] "RemoveContainer" containerID="f5355e523bed9a473601c57987789b58578907b1dfed629d277828f9e3629c25"
Oct 31 22:15:04.572337 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:04.572319    4460 scope.go:122] "RemoveContainer" containerID="1aec227b232f23080f617afe207df5820f0ff495c26bb58a61b573f5624b63fe"
Oct 31 22:15:04.572694 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:04.572668    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr_odh-ai-gateway-infra(e2f44b0f-767e-474e-8f9d-9a84dd2cf281)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr" podUID="e2f44b0f-767e-474e-8f9d-9a84dd2cf281"
Oct 31 22:15:04.574012 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:04.573991    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt" event={"ID":"9bbd45fa-4d2b-422b-8471-8da4428c1a6f","Type":"ContainerStarted","Data":"04d28ea11a917ad2f10ce335368b436f42b271be4c9b94d41ccc2a2ad95653f4"}
Oct 31 22:15:04.576078 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:04.576053    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5" event={"ID":"c6686f63-0d95-4d42-8fcb-861e719277ce","Type":"ContainerStarted","Data":"f4692ee8518acf16544059808cac1d3403ecc31abc5755612ca19254e61887a4"}
Oct 31 22:15:04.578335 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:04.578311    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2" event={"ID":"d68f00ec-71ab-48be-b3bd-d988eeeb5a88","Type":"ContainerStarted","Data":"6fa94befae6bc1824c08b028a3cafcd32e86d205797c289459a64b98d2de70da"}
Oct 31 22:15:04.580307 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:04.580282    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4" event={"ID":"dc0b50fb-08dd-4729-bf9f-01e891bf8ca6","Type":"ContainerStarted","Data":"94b8d1481f7e9bd046c6f0b831d5a8a109ed589d68da14ff3ee91f4d3ab4d4ef"}
Oct 31 22:15:04.582320 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:04.582294    4460 generic.go:356] "Generic (PLEG): container finished" podID="b29dc108-b17e-44be-9f2b-1d133f0acfe7" containerID="c3cd7b0c27daa58819ba3b85a1599f748fc1ca12d477610182d24e7834b381ab" exitCode=60
Oct 31 22:15:04.582433 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:04.582385    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc" event={"ID":"b29dc108-b17e-44be-9f2b-1d133f0acfe7","Type":"ContainerDied","Data":"c3cd7b0c27daa58819ba3b85a1599f748fc1ca12d477610182d24e7834b381ab"}
Oct 31 22:15:04.582625 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:04.582607    4460 scope.go:122] "RemoveContainer" containerID="c3cd7b0c27daa58819ba3b85a1599f748fc1ca12d477610182d24e7834b381ab"
Oct 31 22:15:04.582967 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:04.582910    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc_odh-ai-gateway-infra(b29dc108-b17e-44be-9f2b-1d133f0acfe7)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc" podUID="b29dc108-b17e-44be-9f2b-1d133f0acfe7"
Oct 31 22:15:04.587890 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:04.587868    4460 scope.go:122] "RemoveContainer" containerID="cc56b700ee1366a4d0ce8d4afc2447edf9542948d54eaa5e90f162b2bf47315f"
Oct 31 22:15:04.807767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:04.807717    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt" podStartSLOduration=4.8077020820000005 podStartE2EDuration="4.807702082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 22:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:15:04.806147942 +0000 UTC m=+3544.722457569" watchObservedRunningTime="2026-10-31 22:15:04.807702082 +0000 UTC m=+3544.724011518"
Oct 31 22:15:04.852786 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:04.852734    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd6e81c8-th99j"
Oct 31 22:15:05.013677 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.013649    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4b94e7af-e6fa-434b-b1ac-3069e1d6953e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4b94e7af-e6fa-434b-b1ac-3069e1d6953e-maas-api-openshift-ca\") pod \"4b94e7af-e6fa-434b-b1ac-3069e1d6953e\" (UID: \"4b94e7af-e6fa-434b-b1ac-3069e1d6953e\") "
Oct 31 22:15:05.013838 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.013757    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4b94e7af-e6fa-434b-b1ac-3069e1d6953e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4b94e7af-e6fa-434b-b1ac-3069e1d6953e-maas-api-xks-ca\") pod \"4b94e7af-e6fa-434b-b1ac-3069e1d6953e\" (UID: \"4b94e7af-e6fa-434b-b1ac-3069e1d6953e\") "
Oct 31 22:15:05.013838 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.013829    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4b94e7af-e6fa-434b-b1ac-3069e1d6953e-kube-api-access-fl6r4\" (UniqueName: \"kubernetes.io/projected/4b94e7af-e6fa-434b-b1ac-3069e1d6953e-kube-api-access-fl6r4\") pod \"4b94e7af-e6fa-434b-b1ac-3069e1d6953e\" (UID: \"4b94e7af-e6fa-434b-b1ac-3069e1d6953e\") "
Oct 31 22:15:05.014023 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.013998    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b94e7af-e6fa-434b-b1ac-3069e1d6953e-maas-api-openshift-ca" pod "4b94e7af-e6fa-434b-b1ac-3069e1d6953e" (UID: "4b94e7af-e6fa-434b-b1ac-3069e1d6953e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:15:05.014346 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.014325    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4b94e7af-e6fa-434b-b1ac-3069e1d6953e-maas-api-openshift-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:05.015972 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.015932    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b94e7af-e6fa-434b-b1ac-3069e1d6953e-maas-api-xks-ca" pod "4b94e7af-e6fa-434b-b1ac-3069e1d6953e" (UID: "4b94e7af-e6fa-434b-b1ac-3069e1d6953e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 22:15:05.016086 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.016063    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b94e7af-e6fa-434b-b1ac-3069e1d6953e-kube-api-access-fl6r4" pod "4b94e7af-e6fa-434b-b1ac-3069e1d6953e" (UID: "4b94e7af-e6fa-434b-b1ac-3069e1d6953e"). InnerVolumeSpecName "kube-api-access-fl6r4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:15:05.115730 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.115695    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fl6r4\" (UniqueName: \"kubernetes.io/projected/4b94e7af-e6fa-434b-b1ac-3069e1d6953e-kube-api-access-fl6r4\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:05.115730 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.115726    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4b94e7af-e6fa-434b-b1ac-3069e1d6953e-maas-api-xks-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:05.593956 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.593907    4460 generic.go:356] "Generic (PLEG): container finished" podID="9bbd45fa-4d2b-422b-8471-8da4428c1a6f" containerID="04d28ea11a917ad2f10ce335368b436f42b271be4c9b94d41ccc2a2ad95653f4" exitCode=60
Oct 31 22:15:05.594377 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.593988    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt" event={"ID":"9bbd45fa-4d2b-422b-8471-8da4428c1a6f","Type":"ContainerDied","Data":"04d28ea11a917ad2f10ce335368b436f42b271be4c9b94d41ccc2a2ad95653f4"}
Oct 31 22:15:05.594377 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.594067    4460 scope.go:122] "RemoveContainer" containerID="4774fbe4a4e120572f332326ed471b1fe476daacafdc6c6be5473d1cb45aacb8"
Oct 31 22:15:05.594455 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.594385    4460 scope.go:122] "RemoveContainer" containerID="04d28ea11a917ad2f10ce335368b436f42b271be4c9b94d41ccc2a2ad95653f4"
Oct 31 22:15:05.594775 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:05.594754    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt_odh-ai-gateway-infra(9bbd45fa-4d2b-422b-8471-8da4428c1a6f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt" podUID="9bbd45fa-4d2b-422b-8471-8da4428c1a6f"
Oct 31 22:15:05.597383 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.597355    4460 generic.go:356] "Generic (PLEG): container finished" podID="c6686f63-0d95-4d42-8fcb-861e719277ce" containerID="f4692ee8518acf16544059808cac1d3403ecc31abc5755612ca19254e61887a4" exitCode=60
Oct 31 22:15:05.597523 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.597426    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5" event={"ID":"c6686f63-0d95-4d42-8fcb-861e719277ce","Type":"ContainerDied","Data":"f4692ee8518acf16544059808cac1d3403ecc31abc5755612ca19254e61887a4"}
Oct 31 22:15:05.597732 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.597716    4460 scope.go:122] "RemoveContainer" containerID="f4692ee8518acf16544059808cac1d3403ecc31abc5755612ca19254e61887a4"
Oct 31 22:15:05.598131 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:05.598110    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5_odh-ai-gateway-infra(c6686f63-0d95-4d42-8fcb-861e719277ce)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5" podUID="c6686f63-0d95-4d42-8fcb-861e719277ce"
Oct 31 22:15:05.599703 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.599682    4460 generic.go:356] "Generic (PLEG): container finished" podID="d68f00ec-71ab-48be-b3bd-d988eeeb5a88" containerID="6fa94befae6bc1824c08b028a3cafcd32e86d205797c289459a64b98d2de70da" exitCode=60
Oct 31 22:15:05.599813 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.599772    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2" event={"ID":"d68f00ec-71ab-48be-b3bd-d988eeeb5a88","Type":"ContainerDied","Data":"6fa94befae6bc1824c08b028a3cafcd32e86d205797c289459a64b98d2de70da"}
Oct 31 22:15:05.600088 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.600070    4460 scope.go:122] "RemoveContainer" containerID="6fa94befae6bc1824c08b028a3cafcd32e86d205797c289459a64b98d2de70da"
Oct 31 22:15:05.600400 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:05.600378    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2_odh-ai-gateway-infra(d68f00ec-71ab-48be-b3bd-d988eeeb5a88)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2" podUID="d68f00ec-71ab-48be-b3bd-d988eeeb5a88"
Oct 31 22:15:05.602283 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.602258    4460 generic.go:356] "Generic (PLEG): container finished" podID="dc0b50fb-08dd-4729-bf9f-01e891bf8ca6" containerID="94b8d1481f7e9bd046c6f0b831d5a8a109ed589d68da14ff3ee91f4d3ab4d4ef" exitCode=60
Oct 31 22:15:05.602395 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.602306    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4" event={"ID":"dc0b50fb-08dd-4729-bf9f-01e891bf8ca6","Type":"ContainerDied","Data":"94b8d1481f7e9bd046c6f0b831d5a8a109ed589d68da14ff3ee91f4d3ab4d4ef"}
Oct 31 22:15:05.602602 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.602584    4460 scope.go:122] "RemoveContainer" containerID="94b8d1481f7e9bd046c6f0b831d5a8a109ed589d68da14ff3ee91f4d3ab4d4ef"
Oct 31 22:15:05.602915 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:05.602890    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4_odh-ai-gateway-infra(dc0b50fb-08dd-4729-bf9f-01e891bf8ca6)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4" podUID="dc0b50fb-08dd-4729-bf9f-01e891bf8ca6"
Oct 31 22:15:05.605864 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.605842    4460 scope.go:122] "RemoveContainer" containerID="c3cd7b0c27daa58819ba3b85a1599f748fc1ca12d477610182d24e7834b381ab"
Oct 31 22:15:05.606259 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:05.606164    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc_odh-ai-gateway-infra(b29dc108-b17e-44be-9f2b-1d133f0acfe7)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc" podUID="b29dc108-b17e-44be-9f2b-1d133f0acfe7"
Oct 31 22:15:05.607924 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.607899    4460 generic.go:356] "Generic (PLEG): container finished" podID="42f6904e-6141-41a7-a7c5-ee9c766193d2" containerID="8ad4ef8ff495ef5d5bb835bd62323b595cd0bf5cd1a0e3eeafc173795d146df4" exitCode=60
Oct 31 22:15:05.608057 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.607998    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29891415-tbdkz" event={"ID":"42f6904e-6141-41a7-a7c5-ee9c766193d2","Type":"ContainerDied","Data":"8ad4ef8ff495ef5d5bb835bd62323b595cd0bf5cd1a0e3eeafc173795d146df4"}
Oct 31 22:15:05.608243 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.608228    4460 scope.go:122] "RemoveContainer" containerID="8ad4ef8ff495ef5d5bb835bd62323b595cd0bf5cd1a0e3eeafc173795d146df4"
Oct 31 22:15:05.608514 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:05.608483    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29891415-tbdkz_odh-ai-gateway-infra(42f6904e-6141-41a7-a7c5-ee9c766193d2)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29891415-tbdkz" podUID="42f6904e-6141-41a7-a7c5-ee9c766193d2"
Oct 31 22:15:05.610134 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.610111    4460 generic.go:356] "Generic (PLEG): container finished" podID="ec53414f-1bc8-4065-819c-d85620195d70" containerID="3c97770f1e8269e433faaa85684ac97da7501543316a2d4f510a6c6f26f26902" exitCode=60
Oct 31 22:15:05.610259 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.610235    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5" event={"ID":"ec53414f-1bc8-4065-819c-d85620195d70","Type":"ContainerDied","Data":"3c97770f1e8269e433faaa85684ac97da7501543316a2d4f510a6c6f26f26902"}
Oct 31 22:15:05.610427 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.610405    4460 scope.go:122] "RemoveContainer" containerID="3c97770f1e8269e433faaa85684ac97da7501543316a2d4f510a6c6f26f26902"
Oct 31 22:15:05.610702 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:05.610675    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5_odh-ai-gateway-infra(ec53414f-1bc8-4065-819c-d85620195d70)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5" podUID="ec53414f-1bc8-4065-819c-d85620195d70"
Oct 31 22:15:05.611838 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.611810    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd6e81c8-th99j"
Oct 31 22:15:05.611965 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.611852    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cd6e81c8-th99j" event={"ID":"4b94e7af-e6fa-434b-b1ac-3069e1d6953e","Type":"ContainerDied","Data":"ba5a31b52c3ce61aed13a3bc560d8d0613016ea28624e1ef44962716b21303b1"}
Oct 31 22:15:05.611965 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.611901    4460 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ba5a31b52c3ce61aed13a3bc560d8d0613016ea28624e1ef44962716b21303b1"
Oct 31 22:15:05.614665 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.614646    4460 scope.go:122] "RemoveContainer" containerID="750468a3c57f9ae7d50199b063ef98764c9bfce42b5b057436959f86f4ab19eb"
Oct 31 22:15:05.614742 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.614723    4460 generic.go:356] "Generic (PLEG): container finished" podID="cde3b26f-a0a0-4c0f-835c-cdf613428100" containerID="6bb0cc8e0300ed4c05d3a3d855fbde5a42afc05b45b73e0e5191a82757894c34" exitCode=60
Oct 31 22:15:05.614819 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.614800    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh" event={"ID":"cde3b26f-a0a0-4c0f-835c-cdf613428100","Type":"ContainerDied","Data":"6bb0cc8e0300ed4c05d3a3d855fbde5a42afc05b45b73e0e5191a82757894c34"}
Oct 31 22:15:05.615105 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.615085    4460 scope.go:122] "RemoveContainer" containerID="6bb0cc8e0300ed4c05d3a3d855fbde5a42afc05b45b73e0e5191a82757894c34"
Oct 31 22:15:05.615383 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:05.615362    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh_odh-ai-gateway-infra(cde3b26f-a0a0-4c0f-835c-cdf613428100)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh" podUID="cde3b26f-a0a0-4c0f-835c-cdf613428100"
Oct 31 22:15:05.616810 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.616790    4460 generic.go:356] "Generic (PLEG): container finished" podID="dfc291e0-afb6-4fbd-8314-2e2a36096128" containerID="7e3ea6d913e4e73ce104fa49a10d073547764b53e0d0dd5af578954d1590bf4d" exitCode=60
Oct 31 22:15:05.616918 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.616816    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd" event={"ID":"dfc291e0-afb6-4fbd-8314-2e2a36096128","Type":"ContainerDied","Data":"7e3ea6d913e4e73ce104fa49a10d073547764b53e0d0dd5af578954d1590bf4d"}
Oct 31 22:15:05.617148 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.617129    4460 scope.go:122] "RemoveContainer" containerID="7e3ea6d913e4e73ce104fa49a10d073547764b53e0d0dd5af578954d1590bf4d"
Oct 31 22:15:05.617435 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:05.617408    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd_odh-ai-gateway-infra(dfc291e0-afb6-4fbd-8314-2e2a36096128)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd" podUID="dfc291e0-afb6-4fbd-8314-2e2a36096128"
Oct 31 22:15:05.618708 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.618693    4460 scope.go:122] "RemoveContainer" containerID="1aec227b232f23080f617afe207df5820f0ff495c26bb58a61b573f5624b63fe"
Oct 31 22:15:05.618907 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:05.618890    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr_odh-ai-gateway-infra(e2f44b0f-767e-474e-8f9d-9a84dd2cf281)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr" podUID="e2f44b0f-767e-474e-8f9d-9a84dd2cf281"
Oct 31 22:15:05.630272 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.630250    4460 scope.go:122] "RemoveContainer" containerID="ee6bd2a8503af4332cd2b190e7c7e78c63f3b13b7eeb772714666fbd3d545332"
Oct 31 22:15:05.646029 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.646005    4460 scope.go:122] "RemoveContainer" containerID="b0f5734d1211594df55db9c64304547f794efe73d1f6ae6d7e326ef235a63b27"
Oct 31 22:15:05.660622 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.660597    4460 scope.go:122] "RemoveContainer" containerID="40674a7a010de7f79eaaa26d08778461ef381f50cc08753f692042b23a2f11f2"
Oct 31 22:15:05.674537 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.674516    4460 scope.go:122] "RemoveContainer" containerID="e7e0098577c224bf355aab3f415a5b22d34952b9a0c1108ea44835048acf3751"
Oct 31 22:15:05.690714 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.690691    4460 scope.go:122] "RemoveContainer" containerID="42755992be5a9873f028f90098550fb3ea0c361b19cdcb5adce5637b0e14b05c"
Oct 31 22:15:05.705971 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:05.705916    4460 scope.go:122] "RemoveContainer" containerID="859e425ac014d253b709645b20dbffc3d952b9fc5996a801dae95bfb485cf94c"
Oct 31 22:15:06.121585 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:06.121547    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:06.628623 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:06.628595    4460 scope.go:122] "RemoveContainer" containerID="8ad4ef8ff495ef5d5bb835bd62323b595cd0bf5cd1a0e3eeafc173795d146df4"
Oct 31 22:15:06.629101 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:06.628988    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29891415-tbdkz_odh-ai-gateway-infra(42f6904e-6141-41a7-a7c5-ee9c766193d2)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29891415-tbdkz" podUID="42f6904e-6141-41a7-a7c5-ee9c766193d2"
Oct 31 22:15:06.630278 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:06.630257    4460 scope.go:122] "RemoveContainer" containerID="3c97770f1e8269e433faaa85684ac97da7501543316a2d4f510a6c6f26f26902"
Oct 31 22:15:06.630599 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:06.630579    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5_odh-ai-gateway-infra(ec53414f-1bc8-4065-819c-d85620195d70)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5" podUID="ec53414f-1bc8-4065-819c-d85620195d70"
Oct 31 22:15:06.631837 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:06.631821    4460 scope.go:122] "RemoveContainer" containerID="6bb0cc8e0300ed4c05d3a3d855fbde5a42afc05b45b73e0e5191a82757894c34"
Oct 31 22:15:06.632136 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:06.632114    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh_odh-ai-gateway-infra(cde3b26f-a0a0-4c0f-835c-cdf613428100)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh" podUID="cde3b26f-a0a0-4c0f-835c-cdf613428100"
Oct 31 22:15:06.633335 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:06.633318    4460 scope.go:122] "RemoveContainer" containerID="7e3ea6d913e4e73ce104fa49a10d073547764b53e0d0dd5af578954d1590bf4d"
Oct 31 22:15:06.633590 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:06.633567    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd_odh-ai-gateway-infra(dfc291e0-afb6-4fbd-8314-2e2a36096128)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd" podUID="dfc291e0-afb6-4fbd-8314-2e2a36096128"
Oct 31 22:15:06.635029 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:06.635011    4460 scope.go:122] "RemoveContainer" containerID="04d28ea11a917ad2f10ce335368b436f42b271be4c9b94d41ccc2a2ad95653f4"
Oct 31 22:15:06.635272 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:06.635248    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt_odh-ai-gateway-infra(9bbd45fa-4d2b-422b-8471-8da4428c1a6f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt" podUID="9bbd45fa-4d2b-422b-8471-8da4428c1a6f"
Oct 31 22:15:06.636582 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:06.636561    4460 scope.go:122] "RemoveContainer" containerID="f4692ee8518acf16544059808cac1d3403ecc31abc5755612ca19254e61887a4"
Oct 31 22:15:06.636808 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:06.636776    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5_odh-ai-gateway-infra(c6686f63-0d95-4d42-8fcb-861e719277ce)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5" podUID="c6686f63-0d95-4d42-8fcb-861e719277ce"
Oct 31 22:15:06.638304 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:06.638286    4460 scope.go:122] "RemoveContainer" containerID="6fa94befae6bc1824c08b028a3cafcd32e86d205797c289459a64b98d2de70da"
Oct 31 22:15:06.638572 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:06.638551    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2_odh-ai-gateway-infra(d68f00ec-71ab-48be-b3bd-d988eeeb5a88)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2" podUID="d68f00ec-71ab-48be-b3bd-d988eeeb5a88"
Oct 31 22:15:06.639631 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:06.639617    4460 scope.go:122] "RemoveContainer" containerID="94b8d1481f7e9bd046c6f0b831d5a8a109ed589d68da14ff3ee91f4d3ab4d4ef"
Oct 31 22:15:06.639806 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:06.639789    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4_odh-ai-gateway-infra(dc0b50fb-08dd-4729-bf9f-01e891bf8ca6)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4" podUID="dc0b50fb-08dd-4729-bf9f-01e891bf8ca6"
Oct 31 22:15:08.020483 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:08.020441    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:09.818694 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:09.818655    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:09.847480 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:09.847446    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a5affe-78c59c567-c6hdt" podUID="8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:15:11.515971 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:11.515919    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:12.062997 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:12.062904    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a5affe-78c59c567-c6hdt"
Oct 31 22:15:12.224205 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:12.224170    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1-plugins-config-volume\") pod \"8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1\" (UID: \"8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1\") "
Oct 31 22:15:12.224424 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:12.224264    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1-kube-api-access-rwvj4\" (UniqueName: \"kubernetes.io/projected/8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1-kube-api-access-rwvj4\") pod \"8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1\" (UID: \"8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1\") "
Oct 31 22:15:12.224623 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:12.224594    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1-plugins-config-volume" pod "8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1" (UID: "8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:15:12.225102 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:12.225081    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:12.226391 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:12.226366    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1-kube-api-access-rwvj4" pod "8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1" (UID: "8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1"). InnerVolumeSpecName "kube-api-access-rwvj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:15:12.326379 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:12.326313    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwvj4\" (UniqueName: \"kubernetes.io/projected/8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1-kube-api-access-rwvj4\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:12.691917 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:12.691840    4460 generic.go:356] "Generic (PLEG): container finished" podID="8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1" containerID="4574ea54ca431d88627d1a74c901ebbed08beafcd9a87dc16f9176aa627dd7f7" exitCode=0
Oct 31 22:15:12.691917 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:12.691889    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a5affe-78c59c567-c6hdt"
Oct 31 22:15:12.692409 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:12.691908    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a5affe-78c59c567-c6hdt" event={"ID":"8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1","Type":"ContainerDied","Data":"4574ea54ca431d88627d1a74c901ebbed08beafcd9a87dc16f9176aa627dd7f7"}
Oct 31 22:15:12.692409 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:12.691989    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-a5affe-78c59c567-c6hdt" event={"ID":"8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1","Type":"ContainerDied","Data":"421b614a416e51169d6ceb4ef1da87f1fe9df32cd4495b92be57efecaab345d7"}
Oct 31 22:15:12.692409 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:12.692022    4460 scope.go:122] "RemoveContainer" containerID="4574ea54ca431d88627d1a74c901ebbed08beafcd9a87dc16f9176aa627dd7f7"
Oct 31 22:15:12.702919 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:12.702887    4460 scope.go:122] "RemoveContainer" containerID="4574ea54ca431d88627d1a74c901ebbed08beafcd9a87dc16f9176aa627dd7f7"
Oct 31 22:15:12.703230 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:12.703211    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4574ea54ca431d88627d1a74c901ebbed08beafcd9a87dc16f9176aa627dd7f7\": container with ID starting with 4574ea54ca431d88627d1a74c901ebbed08beafcd9a87dc16f9176aa627dd7f7 not found: ID does not exist" containerID="4574ea54ca431d88627d1a74c901ebbed08beafcd9a87dc16f9176aa627dd7f7"
Oct 31 22:15:12.703298 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:12.703238    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4574ea54ca431d88627d1a74c901ebbed08beafcd9a87dc16f9176aa627dd7f7"} err="failed to get container status \"4574ea54ca431d88627d1a74c901ebbed08beafcd9a87dc16f9176aa627dd7f7\": rpc error: code = NotFound desc = could not find container \"4574ea54ca431d88627d1a74c901ebbed08beafcd9a87dc16f9176aa627dd7f7\": container with ID starting with 4574ea54ca431d88627d1a74c901ebbed08beafcd9a87dc16f9176aa627dd7f7 not found: ID does not exist"
Oct 31 22:15:12.752916 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:12.752876    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-a5affe-78c59c567-c6hdt"]
Oct 31 22:15:12.764281 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:12.764253    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-a5affe-78c59c567-c6hdt"]
Oct 31 22:15:13.362528 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:13.362491    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:14.227643 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:14.227609    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1" path="/var/lib/kubelet/pods/8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1/volumes"
Oct 31 22:15:14.609188 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:14.609157    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 22:15:14.875433 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:14.875345    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:15.222143 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:15.220061    4460 scope.go:122] "RemoveContainer" containerID="c3cd7b0c27daa58819ba3b85a1599f748fc1ca12d477610182d24e7834b381ab"
Oct 31 22:15:15.579010 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:15.578974    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:15.708036 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:15.707835    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:15.829705 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:15.829625    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:16.218000 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:16.217911    4460 scope.go:122] "RemoveContainer" containerID="6fa94befae6bc1824c08b028a3cafcd32e86d205797c289459a64b98d2de70da"
Oct 31 22:15:16.229409 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:16.229364    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:16.763250 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:16.763214    4460 generic.go:356] "Generic (PLEG): container finished" podID="b29dc108-b17e-44be-9f2b-1d133f0acfe7" containerID="96235cd6531713450ca27f4722879a36b8175d4b2c4c5f285e0a9ac3872211f5" exitCode=60
Oct 31 22:15:16.763783 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:16.763296    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc" event={"ID":"b29dc108-b17e-44be-9f2b-1d133f0acfe7","Type":"ContainerDied","Data":"96235cd6531713450ca27f4722879a36b8175d4b2c4c5f285e0a9ac3872211f5"}
Oct 31 22:15:16.763783 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:16.763409    4460 scope.go:122] "RemoveContainer" containerID="c3cd7b0c27daa58819ba3b85a1599f748fc1ca12d477610182d24e7834b381ab"
Oct 31 22:15:16.763980 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:16.763924    4460 scope.go:122] "RemoveContainer" containerID="96235cd6531713450ca27f4722879a36b8175d4b2c4c5f285e0a9ac3872211f5"
Oct 31 22:15:16.764449 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:16.764419    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc_odh-ai-gateway-infra(b29dc108-b17e-44be-9f2b-1d133f0acfe7)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc" podUID="b29dc108-b17e-44be-9f2b-1d133f0acfe7"
Oct 31 22:15:16.766202 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:16.766182    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2" event={"ID":"d68f00ec-71ab-48be-b3bd-d988eeeb5a88","Type":"ContainerStarted","Data":"15e26d87577eeb77ce351f668157c9319f8cf11d04902e1c224ae47e61b6507d"}
Oct 31 22:15:17.087881 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:17.087789    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:17.219004 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:17.218709    4460 scope.go:122] "RemoveContainer" containerID="94b8d1481f7e9bd046c6f0b831d5a8a109ed589d68da14ff3ee91f4d3ab4d4ef"
Oct 31 22:15:17.223417 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:17.219729    4460 scope.go:122] "RemoveContainer" containerID="f4692ee8518acf16544059808cac1d3403ecc31abc5755612ca19254e61887a4"
Oct 31 22:15:17.776476 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:17.776440    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5" event={"ID":"c6686f63-0d95-4d42-8fcb-861e719277ce","Type":"ContainerStarted","Data":"63b1d34849fab9edcddbc57229d017d084251adab059f0d60b17a6f656a12e67"}
Oct 31 22:15:17.777144 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:17.777108    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:17.778693 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:17.778670    4460 generic.go:356] "Generic (PLEG): container finished" podID="d68f00ec-71ab-48be-b3bd-d988eeeb5a88" containerID="15e26d87577eeb77ce351f668157c9319f8cf11d04902e1c224ae47e61b6507d" exitCode=60
Oct 31 22:15:17.778803 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:17.778734    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2" event={"ID":"d68f00ec-71ab-48be-b3bd-d988eeeb5a88","Type":"ContainerDied","Data":"15e26d87577eeb77ce351f668157c9319f8cf11d04902e1c224ae47e61b6507d"}
Oct 31 22:15:17.778803 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:17.778792    4460 scope.go:122] "RemoveContainer" containerID="6fa94befae6bc1824c08b028a3cafcd32e86d205797c289459a64b98d2de70da"
Oct 31 22:15:17.779201 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:17.779183    4460 scope.go:122] "RemoveContainer" containerID="15e26d87577eeb77ce351f668157c9319f8cf11d04902e1c224ae47e61b6507d"
Oct 31 22:15:17.779597 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:17.779573    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2_odh-ai-gateway-infra(d68f00ec-71ab-48be-b3bd-d988eeeb5a88)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2" podUID="d68f00ec-71ab-48be-b3bd-d988eeeb5a88"
Oct 31 22:15:17.780843 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:17.780818    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4" event={"ID":"dc0b50fb-08dd-4729-bf9f-01e891bf8ca6","Type":"ContainerStarted","Data":"9aa6db4035ce6bc59d2c984cb7fb67795582b1db6152b2faaa7bee45456783a5"}
Oct 31 22:15:17.865031 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:17.852308    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc"]
Oct 31 22:15:17.883745 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:17.875756    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2"]
Oct 31 22:15:18.057453 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:18.057395    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc"
Oct 31 22:15:18.217549 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:18.217504    4460 scope.go:122] "RemoveContainer" containerID="1aec227b232f23080f617afe207df5820f0ff495c26bb58a61b573f5624b63fe"
Oct 31 22:15:18.217734 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:18.217654    4460 scope.go:122] "RemoveContainer" containerID="7e3ea6d913e4e73ce104fa49a10d073547764b53e0d0dd5af578954d1590bf4d"
Oct 31 22:15:18.217825 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:18.217807    4460 scope.go:122] "RemoveContainer" containerID="6bb0cc8e0300ed4c05d3a3d855fbde5a42afc05b45b73e0e5191a82757894c34"
Oct 31 22:15:18.240302 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:18.240266    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b29dc108-b17e-44be-9f2b-1d133f0acfe7-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b29dc108-b17e-44be-9f2b-1d133f0acfe7-maas-api-service-ca\") pod \"b29dc108-b17e-44be-9f2b-1d133f0acfe7\" (UID: \"b29dc108-b17e-44be-9f2b-1d133f0acfe7\") "
Oct 31 22:15:18.240635 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:18.240610    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b29dc108-b17e-44be-9f2b-1d133f0acfe7-maas-api-service-ca" pod "b29dc108-b17e-44be-9f2b-1d133f0acfe7" (UID: "b29dc108-b17e-44be-9f2b-1d133f0acfe7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:15:18.240900 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:18.240878    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b29dc108-b17e-44be-9f2b-1d133f0acfe7-kube-api-access-6v48h\" (UniqueName: \"kubernetes.io/projected/b29dc108-b17e-44be-9f2b-1d133f0acfe7-kube-api-access-6v48h\") pod \"b29dc108-b17e-44be-9f2b-1d133f0acfe7\" (UID: \"b29dc108-b17e-44be-9f2b-1d133f0acfe7\") "
Oct 31 22:15:18.241445 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:18.241420    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b29dc108-b17e-44be-9f2b-1d133f0acfe7-maas-api-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:18.244515 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:18.244489    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b29dc108-b17e-44be-9f2b-1d133f0acfe7-kube-api-access-6v48h" pod "b29dc108-b17e-44be-9f2b-1d133f0acfe7" (UID: "b29dc108-b17e-44be-9f2b-1d133f0acfe7"). InnerVolumeSpecName "kube-api-access-6v48h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:15:18.342660 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:18.342633    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6v48h\" (UniqueName: \"kubernetes.io/projected/b29dc108-b17e-44be-9f2b-1d133f0acfe7-kube-api-access-6v48h\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:18.792376 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:18.792340    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd" event={"ID":"dfc291e0-afb6-4fbd-8314-2e2a36096128","Type":"ContainerStarted","Data":"42b75051662c085130efb66833b2c8507a250e3aa15b645c2a376c318905ec3b"}
Oct 31 22:15:18.794602 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:18.794567    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr" event={"ID":"e2f44b0f-767e-474e-8f9d-9a84dd2cf281","Type":"ContainerStarted","Data":"87106c359b8f7876c3174efd9b0f0508024b4cee2d2cf8ec1aedaae3ffae0abe"}
Oct 31 22:15:18.796647 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:18.796623    4460 generic.go:356] "Generic (PLEG): container finished" podID="c6686f63-0d95-4d42-8fcb-861e719277ce" containerID="63b1d34849fab9edcddbc57229d017d084251adab059f0d60b17a6f656a12e67" exitCode=60
Oct 31 22:15:18.796765 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:18.796691    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5" event={"ID":"c6686f63-0d95-4d42-8fcb-861e719277ce","Type":"ContainerDied","Data":"63b1d34849fab9edcddbc57229d017d084251adab059f0d60b17a6f656a12e67"}
Oct 31 22:15:18.796765 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:18.796746    4460 scope.go:122] "RemoveContainer" containerID="f4692ee8518acf16544059808cac1d3403ecc31abc5755612ca19254e61887a4"
Oct 31 22:15:18.797085 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:18.797069    4460 scope.go:122] "RemoveContainer" containerID="63b1d34849fab9edcddbc57229d017d084251adab059f0d60b17a6f656a12e67"
Oct 31 22:15:18.797421 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:18.797398    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5_odh-ai-gateway-infra(c6686f63-0d95-4d42-8fcb-861e719277ce)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5" podUID="c6686f63-0d95-4d42-8fcb-861e719277ce"
Oct 31 22:15:18.800686 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:18.800655    4460 generic.go:356] "Generic (PLEG): container finished" podID="dc0b50fb-08dd-4729-bf9f-01e891bf8ca6" containerID="9aa6db4035ce6bc59d2c984cb7fb67795582b1db6152b2faaa7bee45456783a5" exitCode=60
Oct 31 22:15:18.800770 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:18.800697    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4" event={"ID":"dc0b50fb-08dd-4729-bf9f-01e891bf8ca6","Type":"ContainerDied","Data":"9aa6db4035ce6bc59d2c984cb7fb67795582b1db6152b2faaa7bee45456783a5"}
Oct 31 22:15:18.801055 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:18.801039    4460 scope.go:122] "RemoveContainer" containerID="9aa6db4035ce6bc59d2c984cb7fb67795582b1db6152b2faaa7bee45456783a5"
Oct 31 22:15:18.801400 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:18.801370    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4_odh-ai-gateway-infra(dc0b50fb-08dd-4729-bf9f-01e891bf8ca6)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4" podUID="dc0b50fb-08dd-4729-bf9f-01e891bf8ca6"
Oct 31 22:15:18.802464 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:18.802446    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc"
Oct 31 22:15:18.802464 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:18.802448    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc" event={"ID":"b29dc108-b17e-44be-9f2b-1d133f0acfe7","Type":"ContainerDied","Data":"24c9f12df86a7f4bfec4f5f8fde73a4cd57ce93dff68205cf2a57036be691314"}
Oct 31 22:15:18.815696 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:18.815675    4460 scope.go:122] "RemoveContainer" containerID="94b8d1481f7e9bd046c6f0b831d5a8a109ed589d68da14ff3ee91f4d3ab4d4ef"
Oct 31 22:15:18.833257 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:18.833228    4460 scope.go:122] "RemoveContainer" containerID="96235cd6531713450ca27f4722879a36b8175d4b2c4c5f285e0a9ac3872211f5"
Oct 31 22:15:18.841457 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:18.841432    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5"]
Oct 31 22:15:18.879138 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:18.879101    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc"]
Oct 31 22:15:18.894615 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:18.894462    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-994336-29891415-9pjgc"]
Oct 31 22:15:18.943173 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:18.943125    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4"]
Oct 31 22:15:18.984891 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:18.984861    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2"
Oct 31 22:15:19.062058 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:19.062031    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d68f00ec-71ab-48be-b3bd-d988eeeb5a88-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d68f00ec-71ab-48be-b3bd-d988eeeb5a88-maas-api-service-ca\") pod \"d68f00ec-71ab-48be-b3bd-d988eeeb5a88\" (UID: \"d68f00ec-71ab-48be-b3bd-d988eeeb5a88\") "
Oct 31 22:15:19.062240 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:19.062228    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d68f00ec-71ab-48be-b3bd-d988eeeb5a88-kube-api-access-x674f\" (UniqueName: \"kubernetes.io/projected/d68f00ec-71ab-48be-b3bd-d988eeeb5a88-kube-api-access-x674f\") pod \"d68f00ec-71ab-48be-b3bd-d988eeeb5a88\" (UID: \"d68f00ec-71ab-48be-b3bd-d988eeeb5a88\") "
Oct 31 22:15:19.062409 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:19.062385    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d68f00ec-71ab-48be-b3bd-d988eeeb5a88-maas-api-service-ca" pod "d68f00ec-71ab-48be-b3bd-d988eeeb5a88" (UID: "d68f00ec-71ab-48be-b3bd-d988eeeb5a88"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:15:19.062713 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:19.062697    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d68f00ec-71ab-48be-b3bd-d988eeeb5a88-maas-api-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:19.064371 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:19.064347    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d68f00ec-71ab-48be-b3bd-d988eeeb5a88-kube-api-access-x674f" pod "d68f00ec-71ab-48be-b3bd-d988eeeb5a88" (UID: "d68f00ec-71ab-48be-b3bd-d988eeeb5a88"). InnerVolumeSpecName "kube-api-access-x674f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:15:19.164076 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:19.163993    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x674f\" (UniqueName: \"kubernetes.io/projected/d68f00ec-71ab-48be-b3bd-d988eeeb5a88-kube-api-access-x674f\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:19.218095 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:19.218057    4460 scope.go:122] "RemoveContainer" containerID="8ad4ef8ff495ef5d5bb835bd62323b595cd0bf5cd1a0e3eeafc173795d146df4"
Oct 31 22:15:19.218269 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:19.218239    4460 scope.go:122] "RemoveContainer" containerID="04d28ea11a917ad2f10ce335368b436f42b271be4c9b94d41ccc2a2ad95653f4"
Oct 31 22:15:19.218407 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:19.218387    4460 scope.go:122] "RemoveContainer" containerID="3c97770f1e8269e433faaa85684ac97da7501543316a2d4f510a6c6f26f26902"
Oct 31 22:15:19.553162 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:19.553126    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:19.813458 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:19.813426    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2"
Oct 31 22:15:19.813867 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:19.813429    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2" event={"ID":"d68f00ec-71ab-48be-b3bd-d988eeeb5a88","Type":"ContainerDied","Data":"b1fc41f803566e69bd2af141fbf517faa99af0ea122933f5ad5e443851b063e6"}
Oct 31 22:15:19.813867 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:19.813589    4460 scope.go:122] "RemoveContainer" containerID="15e26d87577eeb77ce351f668157c9319f8cf11d04902e1c224ae47e61b6507d"
Oct 31 22:15:19.819212 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:19.819108    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29891415-tbdkz" event={"ID":"42f6904e-6141-41a7-a7c5-ee9c766193d2","Type":"ContainerStarted","Data":"e01d34672eef51b73c04dbfd09d269d5d60f419f33fe57fc26a42ee111eda0b7"}
Oct 31 22:15:19.821897 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:19.821861    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5" event={"ID":"ec53414f-1bc8-4065-819c-d85620195d70","Type":"ContainerStarted","Data":"77243e8ebea250a79f1b126cee5d473d5b72347dd809f77bbcd14e963bf6d44a"}
Oct 31 22:15:19.826387 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:19.825561    4460 generic.go:356] "Generic (PLEG): container finished" podID="cde3b26f-a0a0-4c0f-835c-cdf613428100" containerID="eacfeb2f502e520917ddd80bc3dfb46e105c08654cb63a0c98163e524a734c24" exitCode=60
Oct 31 22:15:19.826387 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:19.825661    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh" event={"ID":"cde3b26f-a0a0-4c0f-835c-cdf613428100","Type":"ContainerDied","Data":"eacfeb2f502e520917ddd80bc3dfb46e105c08654cb63a0c98163e524a734c24"}
Oct 31 22:15:19.826387 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:19.826174    4460 scope.go:122] "RemoveContainer" containerID="eacfeb2f502e520917ddd80bc3dfb46e105c08654cb63a0c98163e524a734c24"
Oct 31 22:15:19.826588 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:19.826477    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh_odh-ai-gateway-infra(cde3b26f-a0a0-4c0f-835c-cdf613428100)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh" podUID="cde3b26f-a0a0-4c0f-835c-cdf613428100"
Oct 31 22:15:19.829147 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:19.829121    4460 generic.go:356] "Generic (PLEG): container finished" podID="dfc291e0-afb6-4fbd-8314-2e2a36096128" containerID="42b75051662c085130efb66833b2c8507a250e3aa15b645c2a376c318905ec3b" exitCode=60
Oct 31 22:15:19.829256 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:19.829151    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd" event={"ID":"dfc291e0-afb6-4fbd-8314-2e2a36096128","Type":"ContainerDied","Data":"42b75051662c085130efb66833b2c8507a250e3aa15b645c2a376c318905ec3b"}
Oct 31 22:15:19.829665 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:19.829517    4460 scope.go:122] "RemoveContainer" containerID="42b75051662c085130efb66833b2c8507a250e3aa15b645c2a376c318905ec3b"
Oct 31 22:15:19.829833 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:19.829810    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd_odh-ai-gateway-infra(dfc291e0-afb6-4fbd-8314-2e2a36096128)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd" podUID="dfc291e0-afb6-4fbd-8314-2e2a36096128"
Oct 31 22:15:19.831817 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:19.831775    4460 generic.go:356] "Generic (PLEG): container finished" podID="e2f44b0f-767e-474e-8f9d-9a84dd2cf281" containerID="87106c359b8f7876c3174efd9b0f0508024b4cee2d2cf8ec1aedaae3ffae0abe" exitCode=60
Oct 31 22:15:19.831911 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:19.831828    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr" event={"ID":"e2f44b0f-767e-474e-8f9d-9a84dd2cf281","Type":"ContainerDied","Data":"87106c359b8f7876c3174efd9b0f0508024b4cee2d2cf8ec1aedaae3ffae0abe"}
Oct 31 22:15:19.832161 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:19.832145    4460 scope.go:122] "RemoveContainer" containerID="87106c359b8f7876c3174efd9b0f0508024b4cee2d2cf8ec1aedaae3ffae0abe"
Oct 31 22:15:19.832387 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:19.832370    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr_odh-ai-gateway-infra(e2f44b0f-767e-474e-8f9d-9a84dd2cf281)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr" podUID="e2f44b0f-767e-474e-8f9d-9a84dd2cf281"
Oct 31 22:15:19.832867 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:19.832843    4460 scope.go:122] "RemoveContainer" containerID="6bb0cc8e0300ed4c05d3a3d855fbde5a42afc05b45b73e0e5191a82757894c34"
Oct 31 22:15:19.853310 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:19.853281    4460 scope.go:122] "RemoveContainer" containerID="7e3ea6d913e4e73ce104fa49a10d073547764b53e0d0dd5af578954d1590bf4d"
Oct 31 22:15:19.853526 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:19.853503    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd"]
Oct 31 22:15:19.871015 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:19.870995    4460 scope.go:122] "RemoveContainer" containerID="1aec227b232f23080f617afe207df5820f0ff495c26bb58a61b573f5624b63fe"
Oct 31 22:15:19.985379 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:19.985301    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5"
Oct 31 22:15:20.007955 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.007910    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2"]
Oct 31 22:15:20.013878 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.013670    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-5c54fc-29891415-hf8c2"]
Oct 31 22:15:20.062561 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.062535    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4"
Oct 31 22:15:20.063431 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.063408    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr"]
Oct 31 22:15:20.078393 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.078365    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c6686f63-0d95-4d42-8fcb-861e719277ce-kube-api-access-xx9ht\" (UniqueName: \"kubernetes.io/projected/c6686f63-0d95-4d42-8fcb-861e719277ce-kube-api-access-xx9ht\") pod \"c6686f63-0d95-4d42-8fcb-861e719277ce\" (UID: \"c6686f63-0d95-4d42-8fcb-861e719277ce\") "
Oct 31 22:15:20.078559 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.078518    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c6686f63-0d95-4d42-8fcb-861e719277ce-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6686f63-0d95-4d42-8fcb-861e719277ce-maas-api-service-ca\") pod \"c6686f63-0d95-4d42-8fcb-861e719277ce\" (UID: \"c6686f63-0d95-4d42-8fcb-861e719277ce\") "
Oct 31 22:15:20.078633 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.078588    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc0b50fb-08dd-4729-bf9f-01e891bf8ca6-kube-api-access-xdtnt\" (UniqueName: \"kubernetes.io/projected/dc0b50fb-08dd-4729-bf9f-01e891bf8ca6-kube-api-access-xdtnt\") pod \"dc0b50fb-08dd-4729-bf9f-01e891bf8ca6\" (UID: \"dc0b50fb-08dd-4729-bf9f-01e891bf8ca6\") "
Oct 31 22:15:20.078838 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.078816    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dc0b50fb-08dd-4729-bf9f-01e891bf8ca6-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc0b50fb-08dd-4729-bf9f-01e891bf8ca6-maas-api-service-ca\") pod \"dc0b50fb-08dd-4729-bf9f-01e891bf8ca6\" (UID: \"dc0b50fb-08dd-4729-bf9f-01e891bf8ca6\") "
Oct 31 22:15:20.078917 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.078900    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6686f63-0d95-4d42-8fcb-861e719277ce-maas-api-service-ca" pod "c6686f63-0d95-4d42-8fcb-861e719277ce" (UID: "c6686f63-0d95-4d42-8fcb-861e719277ce"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:15:20.079243 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.079217    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc0b50fb-08dd-4729-bf9f-01e891bf8ca6-maas-api-service-ca" pod "dc0b50fb-08dd-4729-bf9f-01e891bf8ca6" (UID: "dc0b50fb-08dd-4729-bf9f-01e891bf8ca6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:15:20.079617 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.079517    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6686f63-0d95-4d42-8fcb-861e719277ce-maas-api-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:20.079617 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.079548    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc0b50fb-08dd-4729-bf9f-01e891bf8ca6-maas-api-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:20.080816 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.080791    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc0b50fb-08dd-4729-bf9f-01e891bf8ca6-kube-api-access-xdtnt" pod "dc0b50fb-08dd-4729-bf9f-01e891bf8ca6" (UID: "dc0b50fb-08dd-4729-bf9f-01e891bf8ca6"). InnerVolumeSpecName "kube-api-access-xdtnt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:15:20.080916 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.080825    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6686f63-0d95-4d42-8fcb-861e719277ce-kube-api-access-xx9ht" pod "c6686f63-0d95-4d42-8fcb-861e719277ce" (UID: "c6686f63-0d95-4d42-8fcb-861e719277ce"). InnerVolumeSpecName "kube-api-access-xx9ht". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:15:20.181132 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.181098    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdtnt\" (UniqueName: \"kubernetes.io/projected/dc0b50fb-08dd-4729-bf9f-01e891bf8ca6-kube-api-access-xdtnt\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:20.181132 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.181129    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xx9ht\" (UniqueName: \"kubernetes.io/projected/c6686f63-0d95-4d42-8fcb-861e719277ce-kube-api-access-xx9ht\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:20.232643 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.232611    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b29dc108-b17e-44be-9f2b-1d133f0acfe7" path="/var/lib/kubelet/pods/b29dc108-b17e-44be-9f2b-1d133f0acfe7/volumes"
Oct 31 22:15:20.233250 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.233229    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d68f00ec-71ab-48be-b3bd-d988eeeb5a88" path="/var/lib/kubelet/pods/d68f00ec-71ab-48be-b3bd-d988eeeb5a88/volumes"
Oct 31 22:15:20.846759 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.846733    4460 generic.go:356] "Generic (PLEG): container finished" podID="9bbd45fa-4d2b-422b-8471-8da4428c1a6f" containerID="ec362e7d6df408a8b9974244ed26c6b046e73f4cca9577d4cf0a902bb229c005" exitCode=60
Oct 31 22:15:20.847191 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.846785    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt" event={"ID":"9bbd45fa-4d2b-422b-8471-8da4428c1a6f","Type":"ContainerDied","Data":"ec362e7d6df408a8b9974244ed26c6b046e73f4cca9577d4cf0a902bb229c005"}
Oct 31 22:15:20.847191 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.846851    4460 scope.go:122] "RemoveContainer" containerID="04d28ea11a917ad2f10ce335368b436f42b271be4c9b94d41ccc2a2ad95653f4"
Oct 31 22:15:20.847327 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.847311    4460 scope.go:122] "RemoveContainer" containerID="ec362e7d6df408a8b9974244ed26c6b046e73f4cca9577d4cf0a902bb229c005"
Oct 31 22:15:20.847747 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:20.847720    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt_odh-ai-gateway-infra(9bbd45fa-4d2b-422b-8471-8da4428c1a6f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt" podUID="9bbd45fa-4d2b-422b-8471-8da4428c1a6f"
Oct 31 22:15:20.848425 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.848406    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5"
Oct 31 22:15:20.848523 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.848447    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5" event={"ID":"c6686f63-0d95-4d42-8fcb-861e719277ce","Type":"ContainerDied","Data":"a2bdb638c941b977933b19d45054e2bbaa02bbcb5eb3e8a030ed4097ca85d0b5"}
Oct 31 22:15:20.851264 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.851246    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4"
Oct 31 22:15:20.851370 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.851299    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4" event={"ID":"dc0b50fb-08dd-4729-bf9f-01e891bf8ca6","Type":"ContainerDied","Data":"d0d79de56d174c263b1cb9dd5ae08b85ab979ba38f2f794de0e07f33e4e0916b"}
Oct 31 22:15:20.853927 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.853900    4460 generic.go:356] "Generic (PLEG): container finished" podID="42f6904e-6141-41a7-a7c5-ee9c766193d2" containerID="e01d34672eef51b73c04dbfd09d269d5d60f419f33fe57fc26a42ee111eda0b7" exitCode=60
Oct 31 22:15:20.854085 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.854015    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29891415-tbdkz" event={"ID":"42f6904e-6141-41a7-a7c5-ee9c766193d2","Type":"ContainerDied","Data":"e01d34672eef51b73c04dbfd09d269d5d60f419f33fe57fc26a42ee111eda0b7"}
Oct 31 22:15:20.854496 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.854476    4460 scope.go:122] "RemoveContainer" containerID="e01d34672eef51b73c04dbfd09d269d5d60f419f33fe57fc26a42ee111eda0b7"
Oct 31 22:15:20.854844 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:20.854818    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29891415-tbdkz_odh-ai-gateway-infra(42f6904e-6141-41a7-a7c5-ee9c766193d2)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29891415-tbdkz" podUID="42f6904e-6141-41a7-a7c5-ee9c766193d2"
Oct 31 22:15:20.857765 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.857745    4460 generic.go:356] "Generic (PLEG): container finished" podID="ec53414f-1bc8-4065-819c-d85620195d70" containerID="77243e8ebea250a79f1b126cee5d473d5b72347dd809f77bbcd14e963bf6d44a" exitCode=60
Oct 31 22:15:20.857866 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.857798    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5" event={"ID":"ec53414f-1bc8-4065-819c-d85620195d70","Type":"ContainerDied","Data":"77243e8ebea250a79f1b126cee5d473d5b72347dd809f77bbcd14e963bf6d44a"}
Oct 31 22:15:20.858160 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.858146    4460 scope.go:122] "RemoveContainer" containerID="77243e8ebea250a79f1b126cee5d473d5b72347dd809f77bbcd14e963bf6d44a"
Oct 31 22:15:20.858423 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:20.858403    4460 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5_odh-ai-gateway-infra(ec53414f-1bc8-4065-819c-d85620195d70)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5" podUID="ec53414f-1bc8-4065-819c-d85620195d70"
Oct 31 22:15:20.869586 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.869561    4460 scope.go:122] "RemoveContainer" containerID="63b1d34849fab9edcddbc57229d017d084251adab059f0d60b17a6f656a12e67"
Oct 31 22:15:20.884634 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.884609    4460 scope.go:122] "RemoveContainer" containerID="9aa6db4035ce6bc59d2c984cb7fb67795582b1db6152b2faaa7bee45456783a5"
Oct 31 22:15:20.897750 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.897730    4460 scope.go:122] "RemoveContainer" containerID="8ad4ef8ff495ef5d5bb835bd62323b595cd0bf5cd1a0e3eeafc173795d146df4"
Oct 31 22:15:20.915526 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.915504    4460 scope.go:122] "RemoveContainer" containerID="3c97770f1e8269e433faaa85684ac97da7501543316a2d4f510a6c6f26f26902"
Oct 31 22:15:20.922231 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.922195    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5"]
Oct 31 22:15:20.949983 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.948013    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5"]
Oct 31 22:15:20.953584 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:20.953556    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-696f9a-29891415-rw5b5"]
Oct 31 22:15:21.014098 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.014064    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr"
Oct 31 22:15:21.059993 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.059904    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29891415-tbdkz"]
Oct 31 22:15:21.095246 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.095159    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e2f44b0f-767e-474e-8f9d-9a84dd2cf281-kube-api-access-6tktq\" (UniqueName: \"kubernetes.io/projected/e2f44b0f-767e-474e-8f9d-9a84dd2cf281-kube-api-access-6tktq\") pod \"e2f44b0f-767e-474e-8f9d-9a84dd2cf281\" (UID: \"e2f44b0f-767e-474e-8f9d-9a84dd2cf281\") "
Oct 31 22:15:21.095466 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.095440    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e2f44b0f-767e-474e-8f9d-9a84dd2cf281-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2f44b0f-767e-474e-8f9d-9a84dd2cf281-maas-api-service-ca\") pod \"e2f44b0f-767e-474e-8f9d-9a84dd2cf281\" (UID: \"e2f44b0f-767e-474e-8f9d-9a84dd2cf281\") "
Oct 31 22:15:21.095868 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.095836    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2f44b0f-767e-474e-8f9d-9a84dd2cf281-maas-api-service-ca" pod "e2f44b0f-767e-474e-8f9d-9a84dd2cf281" (UID: "e2f44b0f-767e-474e-8f9d-9a84dd2cf281"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:15:21.096402 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.096375    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2f44b0f-767e-474e-8f9d-9a84dd2cf281-maas-api-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:21.100057 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.100022    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2f44b0f-767e-474e-8f9d-9a84dd2cf281-kube-api-access-6tktq" pod "e2f44b0f-767e-474e-8f9d-9a84dd2cf281" (UID: "e2f44b0f-767e-474e-8f9d-9a84dd2cf281"). InnerVolumeSpecName "kube-api-access-6tktq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:15:21.122684 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.122660    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd"
Oct 31 22:15:21.144402 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.144377    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh"]
Oct 31 22:15:21.194964 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.190980    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4"]
Oct 31 22:15:21.198448 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.198407    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-cd421c-29891415-kqjt4"]
Oct 31 22:15:21.199292 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.199252    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dfc291e0-afb6-4fbd-8314-2e2a36096128-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfc291e0-afb6-4fbd-8314-2e2a36096128-maas-api-service-ca\") pod \"dfc291e0-afb6-4fbd-8314-2e2a36096128\" (UID: \"dfc291e0-afb6-4fbd-8314-2e2a36096128\") "
Oct 31 22:15:21.199540 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.199515    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dfc291e0-afb6-4fbd-8314-2e2a36096128-kube-api-access-cb499\" (UniqueName: \"kubernetes.io/projected/dfc291e0-afb6-4fbd-8314-2e2a36096128-kube-api-access-cb499\") pod \"dfc291e0-afb6-4fbd-8314-2e2a36096128\" (UID: \"dfc291e0-afb6-4fbd-8314-2e2a36096128\") "
Oct 31 22:15:21.199757 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.199633    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfc291e0-afb6-4fbd-8314-2e2a36096128-maas-api-service-ca" pod "dfc291e0-afb6-4fbd-8314-2e2a36096128" (UID: "dfc291e0-afb6-4fbd-8314-2e2a36096128"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:15:21.200840 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.200811    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tktq\" (UniqueName: \"kubernetes.io/projected/e2f44b0f-767e-474e-8f9d-9a84dd2cf281-kube-api-access-6tktq\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:21.201015 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.200997    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfc291e0-afb6-4fbd-8314-2e2a36096128-maas-api-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:21.210073 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.207452    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfc291e0-afb6-4fbd-8314-2e2a36096128-kube-api-access-cb499" pod "dfc291e0-afb6-4fbd-8314-2e2a36096128" (UID: "dfc291e0-afb6-4fbd-8314-2e2a36096128"). InnerVolumeSpecName "kube-api-access-cb499". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:15:21.303986 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.303935    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cb499\" (UniqueName: \"kubernetes.io/projected/dfc291e0-afb6-4fbd-8314-2e2a36096128-kube-api-access-cb499\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:21.309548 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.309522    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh"
Oct 31 22:15:21.405461 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.405373    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cde3b26f-a0a0-4c0f-835c-cdf613428100-kube-api-access-s9z5q\" (UniqueName: \"kubernetes.io/projected/cde3b26f-a0a0-4c0f-835c-cdf613428100-kube-api-access-s9z5q\") pod \"cde3b26f-a0a0-4c0f-835c-cdf613428100\" (UID: \"cde3b26f-a0a0-4c0f-835c-cdf613428100\") "
Oct 31 22:15:21.405461 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.405432    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cde3b26f-a0a0-4c0f-835c-cdf613428100-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cde3b26f-a0a0-4c0f-835c-cdf613428100-maas-api-service-ca\") pod \"cde3b26f-a0a0-4c0f-835c-cdf613428100\" (UID: \"cde3b26f-a0a0-4c0f-835c-cdf613428100\") "
Oct 31 22:15:21.405840 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.405798    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cde3b26f-a0a0-4c0f-835c-cdf613428100-maas-api-service-ca" pod "cde3b26f-a0a0-4c0f-835c-cdf613428100" (UID: "cde3b26f-a0a0-4c0f-835c-cdf613428100"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:15:21.406571 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.406543    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cde3b26f-a0a0-4c0f-835c-cdf613428100-maas-api-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:21.407719 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.407694    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cde3b26f-a0a0-4c0f-835c-cdf613428100-kube-api-access-s9z5q" pod "cde3b26f-a0a0-4c0f-835c-cdf613428100" (UID: "cde3b26f-a0a0-4c0f-835c-cdf613428100"). InnerVolumeSpecName "kube-api-access-s9z5q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:15:21.455001 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:21.454966    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:15:21.508097 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.508049    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9z5q\" (UniqueName: \"kubernetes.io/projected/cde3b26f-a0a0-4c0f-835c-cdf613428100-kube-api-access-s9z5q\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:21.584396 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.584349    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:21.867388 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.867353    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh"
Oct 31 22:15:21.867882 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.867355    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh" event={"ID":"cde3b26f-a0a0-4c0f-835c-cdf613428100","Type":"ContainerDied","Data":"f7f85d75252f1763575001e9ff71469ac5d6949853aff89fff07b8ee5139ff60"}
Oct 31 22:15:21.867882 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.867494    4460 scope.go:122] "RemoveContainer" containerID="eacfeb2f502e520917ddd80bc3dfb46e105c08654cb63a0c98163e524a734c24"
Oct 31 22:15:21.869351 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.869327    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd"
Oct 31 22:15:21.869491 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.869345    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd" event={"ID":"dfc291e0-afb6-4fbd-8314-2e2a36096128","Type":"ContainerDied","Data":"fd8c953277fcdb7e150b2a9bff4e7898ca533c1b4c586c035651b7455df23ecd"}
Oct 31 22:15:21.871217 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.871194    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr"
Oct 31 22:15:21.871648 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.871203    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr" event={"ID":"e2f44b0f-767e-474e-8f9d-9a84dd2cf281","Type":"ContainerDied","Data":"85853de135d1a09ab98c56f6dfc8bf1aef11118e42bd18625488573649c439da"}
Oct 31 22:15:21.882375 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.882258    4460 scope.go:122] "RemoveContainer" containerID="42b75051662c085130efb66833b2c8507a250e3aa15b645c2a376c318905ec3b"
Oct 31 22:15:21.897444 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:21.897423    4460 scope.go:122] "RemoveContainer" containerID="87106c359b8f7876c3174efd9b0f0508024b4cee2d2cf8ec1aedaae3ffae0abe"
Oct 31 22:15:22.056070 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.056034    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt"]
Oct 31 22:15:22.065336 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.065260    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd"]
Oct 31 22:15:22.073254 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.073223    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a73b1-29891415-hq6nd"]
Oct 31 22:15:22.088591 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.088555    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5"
Oct 31 22:15:22.111195 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.111169    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh"]
Oct 31 22:15:22.122278 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.122198    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec53414f-1bc8-4065-819c-d85620195d70-kube-api-access-txbmg\" (UniqueName: \"kubernetes.io/projected/ec53414f-1bc8-4065-819c-d85620195d70-kube-api-access-txbmg\") pod \"ec53414f-1bc8-4065-819c-d85620195d70\" (UID: \"ec53414f-1bc8-4065-819c-d85620195d70\") "
Oct 31 22:15:22.122278 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.122272    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ec53414f-1bc8-4065-819c-d85620195d70-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec53414f-1bc8-4065-819c-d85620195d70-maas-api-service-ca\") pod \"ec53414f-1bc8-4065-819c-d85620195d70\" (UID: \"ec53414f-1bc8-4065-819c-d85620195d70\") "
Oct 31 22:15:22.123494 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.123466    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec53414f-1bc8-4065-819c-d85620195d70-maas-api-service-ca" pod "ec53414f-1bc8-4065-819c-d85620195d70" (UID: "ec53414f-1bc8-4065-819c-d85620195d70"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:15:22.126314 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.126285    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c08360-29891415-z7qzh"]
Oct 31 22:15:22.126931 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.126911    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec53414f-1bc8-4065-819c-d85620195d70-kube-api-access-txbmg" pod "ec53414f-1bc8-4065-819c-d85620195d70" (UID: "ec53414f-1bc8-4065-819c-d85620195d70"). InnerVolumeSpecName "kube-api-access-txbmg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:15:22.145585 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.145346    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29891415-tbdkz"
Oct 31 22:15:22.169080 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.169019    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr"]
Oct 31 22:15:22.176976 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.176910    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-1b0cce-29891415-4x7mr"]
Oct 31 22:15:22.223862 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.223822    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42f6904e-6141-41a7-a7c5-ee9c766193d2-kube-api-access-fsc6q\" (UniqueName: \"kubernetes.io/projected/42f6904e-6141-41a7-a7c5-ee9c766193d2-kube-api-access-fsc6q\") pod \"42f6904e-6141-41a7-a7c5-ee9c766193d2\" (UID: \"42f6904e-6141-41a7-a7c5-ee9c766193d2\") "
Oct 31 22:15:22.224085 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.223968    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/42f6904e-6141-41a7-a7c5-ee9c766193d2-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42f6904e-6141-41a7-a7c5-ee9c766193d2-maas-api-service-ca\") pod \"42f6904e-6141-41a7-a7c5-ee9c766193d2\" (UID: \"42f6904e-6141-41a7-a7c5-ee9c766193d2\") "
Oct 31 22:15:22.224704 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.224676    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txbmg\" (UniqueName: \"kubernetes.io/projected/ec53414f-1bc8-4065-819c-d85620195d70-kube-api-access-txbmg\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:22.224817 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.224708    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec53414f-1bc8-4065-819c-d85620195d70-maas-api-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:22.226240 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.226211    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42f6904e-6141-41a7-a7c5-ee9c766193d2-maas-api-service-ca" pod "42f6904e-6141-41a7-a7c5-ee9c766193d2" (UID: "42f6904e-6141-41a7-a7c5-ee9c766193d2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:15:22.228372 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.228348    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt"
Oct 31 22:15:22.232207 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.232184    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42f6904e-6141-41a7-a7c5-ee9c766193d2-kube-api-access-fsc6q" pod "42f6904e-6141-41a7-a7c5-ee9c766193d2" (UID: "42f6904e-6141-41a7-a7c5-ee9c766193d2"). InnerVolumeSpecName "kube-api-access-fsc6q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:15:22.236994 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.236967    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c6686f63-0d95-4d42-8fcb-861e719277ce" path="/var/lib/kubelet/pods/c6686f63-0d95-4d42-8fcb-861e719277ce/volumes"
Oct 31 22:15:22.237665 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.237643    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cde3b26f-a0a0-4c0f-835c-cdf613428100" path="/var/lib/kubelet/pods/cde3b26f-a0a0-4c0f-835c-cdf613428100/volumes"
Oct 31 22:15:22.238290 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.238270    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dc0b50fb-08dd-4729-bf9f-01e891bf8ca6" path="/var/lib/kubelet/pods/dc0b50fb-08dd-4729-bf9f-01e891bf8ca6/volumes"
Oct 31 22:15:22.239315 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.239295    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dfc291e0-afb6-4fbd-8314-2e2a36096128" path="/var/lib/kubelet/pods/dfc291e0-afb6-4fbd-8314-2e2a36096128/volumes"
Oct 31 22:15:22.239898 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.239879    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e2f44b0f-767e-474e-8f9d-9a84dd2cf281" path="/var/lib/kubelet/pods/e2f44b0f-767e-474e-8f9d-9a84dd2cf281/volumes"
Oct 31 22:15:22.326025 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.325992    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9bbd45fa-4d2b-422b-8471-8da4428c1a6f-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bbd45fa-4d2b-422b-8471-8da4428c1a6f-maas-api-service-ca\") pod \"9bbd45fa-4d2b-422b-8471-8da4428c1a6f\" (UID: \"9bbd45fa-4d2b-422b-8471-8da4428c1a6f\") "
Oct 31 22:15:22.326177 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.326101    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9bbd45fa-4d2b-422b-8471-8da4428c1a6f-kube-api-access-fwrbk\" (UniqueName: \"kubernetes.io/projected/9bbd45fa-4d2b-422b-8471-8da4428c1a6f-kube-api-access-fwrbk\") pod \"9bbd45fa-4d2b-422b-8471-8da4428c1a6f\" (UID: \"9bbd45fa-4d2b-422b-8471-8da4428c1a6f\") "
Oct 31 22:15:22.326338 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.326316    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bbd45fa-4d2b-422b-8471-8da4428c1a6f-maas-api-service-ca" pod "9bbd45fa-4d2b-422b-8471-8da4428c1a6f" (UID: "9bbd45fa-4d2b-422b-8471-8da4428c1a6f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:15:22.326851 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.326834    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fsc6q\" (UniqueName: \"kubernetes.io/projected/42f6904e-6141-41a7-a7c5-ee9c766193d2-kube-api-access-fsc6q\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:22.326896 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.326854    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bbd45fa-4d2b-422b-8471-8da4428c1a6f-maas-api-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:22.326896 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.326866    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/42f6904e-6141-41a7-a7c5-ee9c766193d2-maas-api-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:22.328253 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.328234    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bbd45fa-4d2b-422b-8471-8da4428c1a6f-kube-api-access-fwrbk" pod "9bbd45fa-4d2b-422b-8471-8da4428c1a6f" (UID: "9bbd45fa-4d2b-422b-8471-8da4428c1a6f"). InnerVolumeSpecName "kube-api-access-fwrbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:15:22.427962 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.427864    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwrbk\" (UniqueName: \"kubernetes.io/projected/9bbd45fa-4d2b-422b-8471-8da4428c1a6f-kube-api-access-fwrbk\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:22.893315 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.893285    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt"
Oct 31 22:15:22.893710 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.893317    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt" event={"ID":"9bbd45fa-4d2b-422b-8471-8da4428c1a6f","Type":"ContainerDied","Data":"486f373479888e14cbac2436cab39d9d77350e59808390311b830837a0c218fe"}
Oct 31 22:15:22.893710 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.893377    4460 scope.go:122] "RemoveContainer" containerID="ec362e7d6df408a8b9974244ed26c6b046e73f4cca9577d4cf0a902bb229c005"
Oct 31 22:15:22.894857 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.894834    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29891415-tbdkz"
Oct 31 22:15:22.894979 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.894832    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29891415-tbdkz" event={"ID":"42f6904e-6141-41a7-a7c5-ee9c766193d2","Type":"ContainerDied","Data":"3a121fd3c0cbffc66819a72b718dc11a43af9aa9fdeb7e9c9029588fe1b1fbc9"}
Oct 31 22:15:22.896534 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.896500    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5"
Oct 31 22:15:22.896689 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.896580    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5" event={"ID":"ec53414f-1bc8-4065-819c-d85620195d70","Type":"ContainerDied","Data":"311eb5916d8b489116bc192ae79c05cd5c78ac81597f6d114f841ce3cccaaaa5"}
Oct 31 22:15:22.906634 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.906612    4460 scope.go:122] "RemoveContainer" containerID="e01d34672eef51b73c04dbfd09d269d5d60f419f33fe57fc26a42ee111eda0b7"
Oct 31 22:15:22.916352 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.916332    4460 scope.go:122] "RemoveContainer" containerID="77243e8ebea250a79f1b126cee5d473d5b72347dd809f77bbcd14e963bf6d44a"
Oct 31 22:15:22.956863 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.956841    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5"]
Oct 31 22:15:22.970721 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:22.970698    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-943963-29891415-pp9f5"]
Oct 31 22:15:23.007355 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:23.007328    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29891415-tbdkz"]
Oct 31 22:15:23.010921 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:23.010897    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29891415-tbdkz"]
Oct 31 22:15:23.047519 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:23.047492    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt"]
Oct 31 22:15:23.050912 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:23.050892    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-710927-29891415-zfwqt"]
Oct 31 22:15:23.228027 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:23.227933    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:23.349238 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:23.349201    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:23.584089 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:23.584008    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:24.035864 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:24.035827    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:24.228056 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:24.228020    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="42f6904e-6141-41a7-a7c5-ee9c766193d2" path="/var/lib/kubelet/pods/42f6904e-6141-41a7-a7c5-ee9c766193d2/volumes"
Oct 31 22:15:24.228513 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:24.228499    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9bbd45fa-4d2b-422b-8471-8da4428c1a6f" path="/var/lib/kubelet/pods/9bbd45fa-4d2b-422b-8471-8da4428c1a6f/volumes"
Oct 31 22:15:24.228969 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:24.228954    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ec53414f-1bc8-4065-819c-d85620195d70" path="/var/lib/kubelet/pods/ec53414f-1bc8-4065-819c-d85620195d70/volumes"
Oct 31 22:15:24.440709 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:24.440617    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:24.549096 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:24.549059    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:25.281237 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:25.281190    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:27.044889 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:27.044853    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:27.863572 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:27.863534    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-710927-5bdd596d55-2t8rq"]
Oct 31 22:15:27.976773 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:27.976631    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:28.106890 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:28.106849    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:28.851879 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:28.851839    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:30.612133 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:30.612090    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:32.361306 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:32.361264    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:32.864034 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:32.863986    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-710927-5bdd596d55-2t8rq" podUID="5953786a-52ca-4d85-a60d-a276d6889552" containerName="payload-pre-processing" containerID="cri-o://43a7df353f4b45d7f732e704f2e48a98660f5293fdfa06142293c0ddf1214d8f" gracePeriod=25
Oct 31 22:15:34.045487 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:34.045451    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:34.055621 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:34.055578    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-710927-5bdd596d55-2t8rq" podUID="5953786a-52ca-4d85-a60d-a276d6889552" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:15:34.576403 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:34.576364    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-6b5dab-75bfb5c8df-nh5kr"]
Oct 31 22:15:34.613922 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:34.613886    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-6b5dab-59c4c79ff9-n89dv"]
Oct 31 22:15:34.812451 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:34.812393    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:34.919804 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:34.919713    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:35.850611 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:35.850561    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:37.640847 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:37.640809    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:39.434560 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:39.434529    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:39.576727 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:39.576667    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-6b5dab-75bfb5c8df-nh5kr" podUID="afe24009-66f9-4972-9e31-709ec4fa2acb" containerName="payload-processing" containerID="cri-o://ff31202536e63724b16d8793d4bee396ff047b09e5ba5cc72974944c7baa7c23" gracePeriod=25
Oct 31 22:15:39.614516 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:39.614473    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-6b5dab-59c4c79ff9-n89dv" podUID="15a6d325-3256-4d12-b698-fffafe8c39f0" containerName="payload-pre-processing" containerID="cri-o://c826d9add18ef144496b83592c85332bf74d18da80d5972625ebb8ed5493ba9a" gracePeriod=25
Oct 31 22:15:39.912731 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:39.912686    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-6b5dab-59c4c79ff9-n89dv" podUID="15a6d325-3256-4d12-b698-fffafe8c39f0" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:15:39.912731 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:39.912709    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-6b5dab-75bfb5c8df-nh5kr" podUID="afe24009-66f9-4972-9e31-709ec4fa2acb" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:15:41.197669 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:41.197625    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:42.708647 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:42.708616    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:43.275001 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:43.274928    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:43.387650 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:43.387615    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:43.506101 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:43.506060    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:43.878478 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:43.878437    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:44.056155 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:44.056117    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-710927-5bdd596d55-2t8rq" podUID="5953786a-52ca-4d85-a60d-a276d6889552" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:15:45.216479 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:45.216440    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:46.460824 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:46.460789    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:46.573057 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:46.573023    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:47.018261 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:47.018222    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:48.785387 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:48.785348    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:49.912275 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:49.912223    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-6b5dab-59c4c79ff9-n89dv" podUID="15a6d325-3256-4d12-b698-fffafe8c39f0" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:15:49.912687 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:49.912236    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-6b5dab-75bfb5c8df-nh5kr" podUID="afe24009-66f9-4972-9e31-709ec4fa2acb" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:15:50.601811 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:50.601773    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:50.888243 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:50.888147    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:51.002368 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:51.002327    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:51.122147 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:51.122110    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:51.194314 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:51.194235    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"]
Oct 31 22:15:51.194516 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:51.194486    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs" podUID="2f79cdb1-ad35-4719-8f2b-591fa5965bc5" containerName="istio-proxy" containerID="cri-o://d2c27a2e1df4c27f2a6f001cad7fe59f79c1ffcd718cef12018b0a61c4f2067d" gracePeriod=30
Oct 31 22:15:51.239594 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:51.239561    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:51.465685 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:51.465606    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:15:52.556685 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:52.556644    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:53.135968 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:53.135891    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-710927-5bdd596d55-2t8rq"
Oct 31 22:15:53.255698 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:53.255654    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5953786a-52ca-4d85-a60d-a276d6889552-kube-api-access-r468v\" (UniqueName: \"kubernetes.io/projected/5953786a-52ca-4d85-a60d-a276d6889552-kube-api-access-r468v\") pod \"5953786a-52ca-4d85-a60d-a276d6889552\" (UID: \"5953786a-52ca-4d85-a60d-a276d6889552\") "
Oct 31 22:15:53.255883 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:53.255848    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5953786a-52ca-4d85-a60d-a276d6889552-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5953786a-52ca-4d85-a60d-a276d6889552-plugins-config-volume\") pod \"5953786a-52ca-4d85-a60d-a276d6889552\" (UID: \"5953786a-52ca-4d85-a60d-a276d6889552\") "
Oct 31 22:15:53.256300 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:53.256265    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5953786a-52ca-4d85-a60d-a276d6889552-plugins-config-volume" pod "5953786a-52ca-4d85-a60d-a276d6889552" (UID: "5953786a-52ca-4d85-a60d-a276d6889552"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:15:53.256597 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:53.256577    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5953786a-52ca-4d85-a60d-a276d6889552-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:53.257838 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:53.257814    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5953786a-52ca-4d85-a60d-a276d6889552-kube-api-access-r468v" pod "5953786a-52ca-4d85-a60d-a276d6889552" (UID: "5953786a-52ca-4d85-a60d-a276d6889552"). InnerVolumeSpecName "kube-api-access-r468v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:15:53.358455 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:53.358427    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r468v\" (UniqueName: \"kubernetes.io/projected/5953786a-52ca-4d85-a60d-a276d6889552-kube-api-access-r468v\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:53.373204 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:53.373173    4460 generic.go:356] "Generic (PLEG): container finished" podID="5953786a-52ca-4d85-a60d-a276d6889552" containerID="43a7df353f4b45d7f732e704f2e48a98660f5293fdfa06142293c0ddf1214d8f" exitCode=0
Oct 31 22:15:53.373338 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:53.373222    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-710927-5bdd596d55-2t8rq"
Oct 31 22:15:53.373338 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:53.373228    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-710927-5bdd596d55-2t8rq" event={"ID":"5953786a-52ca-4d85-a60d-a276d6889552","Type":"ContainerDied","Data":"43a7df353f4b45d7f732e704f2e48a98660f5293fdfa06142293c0ddf1214d8f"}
Oct 31 22:15:53.373338 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:53.373273    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-710927-5bdd596d55-2t8rq" event={"ID":"5953786a-52ca-4d85-a60d-a276d6889552","Type":"ContainerDied","Data":"d4d99be858a733888ab88dcbd0fd6facb7901bb7300c4007f2bcce3b7049195f"}
Oct 31 22:15:53.373338 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:53.373301    4460 scope.go:122] "RemoveContainer" containerID="43a7df353f4b45d7f732e704f2e48a98660f5293fdfa06142293c0ddf1214d8f"
Oct 31 22:15:53.385907 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:53.385884    4460 scope.go:122] "RemoveContainer" containerID="43a7df353f4b45d7f732e704f2e48a98660f5293fdfa06142293c0ddf1214d8f"
Oct 31 22:15:53.386279 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:15:53.386222    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43a7df353f4b45d7f732e704f2e48a98660f5293fdfa06142293c0ddf1214d8f\": container with ID starting with 43a7df353f4b45d7f732e704f2e48a98660f5293fdfa06142293c0ddf1214d8f not found: ID does not exist" containerID="43a7df353f4b45d7f732e704f2e48a98660f5293fdfa06142293c0ddf1214d8f"
Oct 31 22:15:53.386279 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:53.386261    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43a7df353f4b45d7f732e704f2e48a98660f5293fdfa06142293c0ddf1214d8f"} err="failed to get container status \"43a7df353f4b45d7f732e704f2e48a98660f5293fdfa06142293c0ddf1214d8f\": rpc error: code = NotFound desc = could not find container \"43a7df353f4b45d7f732e704f2e48a98660f5293fdfa06142293c0ddf1214d8f\": container with ID starting with 43a7df353f4b45d7f732e704f2e48a98660f5293fdfa06142293c0ddf1214d8f not found: ID does not exist"
Oct 31 22:15:53.433744 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:53.433714    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-710927-5bdd596d55-2t8rq"]
Oct 31 22:15:53.437795 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:53.437771    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-710927-5bdd596d55-2t8rq"]
Oct 31 22:15:54.227284 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:54.227249    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5953786a-52ca-4d85-a60d-a276d6889552" path="/var/lib/kubelet/pods/5953786a-52ca-4d85-a60d-a276d6889552/volumes"
Oct 31 22:15:54.471806 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:54.471767    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:56.245374 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:56.245344    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:56.402079 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:56.401697    4460 generic.go:356] "Generic (PLEG): container finished" podID="2f79cdb1-ad35-4719-8f2b-591fa5965bc5" containerID="d2c27a2e1df4c27f2a6f001cad7fe59f79c1ffcd718cef12018b0a61c4f2067d" exitCode=0
Oct 31 22:15:56.402079 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:56.401885    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs" event={"ID":"2f79cdb1-ad35-4719-8f2b-591fa5965bc5","Type":"ContainerDied","Data":"d2c27a2e1df4c27f2a6f001cad7fe59f79c1ffcd718cef12018b0a61c4f2067d"}
Oct 31 22:15:56.402079 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:56.401964    4460 scope.go:122] "RemoveContainer" containerID="0246258b62f25177b9d52adc7f42966be057d5ba2aac2bae130835c6c169f299"
Oct 31 22:15:56.520024 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:56.519999    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:15:56.614090 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:56.614058    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-workload-socket\") pod \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") "
Oct 31 22:15:56.614302 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:56.614106    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-kube-api-access-xsc8d\" (UniqueName: \"kubernetes.io/projected/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-kube-api-access-xsc8d\") pod \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") "
Oct 31 22:15:56.614302 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:56.614142    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istio-token\" (UniqueName: \"kubernetes.io/projected/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istio-token\") pod \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") "
Oct 31 22:15:56.614302 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:56.614201    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istio-envoy\") pod \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") "
Oct 31 22:15:56.614495 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:56.614309    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istio-data\") pod \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") "
Oct 31 22:15:56.614495 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:56.614368    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-workload-socket" pod "2f79cdb1-ad35-4719-8f2b-591fa5965bc5" (UID: "2f79cdb1-ad35-4719-8f2b-591fa5965bc5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:15:56.614495 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:56.614396    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istio-podinfo\") pod \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") "
Oct 31 22:15:56.614766 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:56.614573    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istiod-ca-cert\") pod \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") "
Oct 31 22:15:56.614766 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:56.614634    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-credential-socket\") pod \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") "
Oct 31 22:15:56.614766 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:56.614723    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-workload-certs\") pod \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\" (UID: \"2f79cdb1-ad35-4719-8f2b-591fa5965bc5\") "
Oct 31 22:15:56.615527 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:56.615497    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:56.615650 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:56.615604    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-credential-socket" pod "2f79cdb1-ad35-4719-8f2b-591fa5965bc5" (UID: "2f79cdb1-ad35-4719-8f2b-591fa5965bc5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:15:56.615779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:56.615739    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-workload-certs" pod "2f79cdb1-ad35-4719-8f2b-591fa5965bc5" (UID: "2f79cdb1-ad35-4719-8f2b-591fa5965bc5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:15:56.615853 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:56.615834    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istiod-ca-cert" pod "2f79cdb1-ad35-4719-8f2b-591fa5965bc5" (UID: "2f79cdb1-ad35-4719-8f2b-591fa5965bc5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:15:56.615916 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:56.615662    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istio-data" pod "2f79cdb1-ad35-4719-8f2b-591fa5965bc5" (UID: "2f79cdb1-ad35-4719-8f2b-591fa5965bc5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:15:56.618142 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:56.618114    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-kube-api-access-xsc8d" pod "2f79cdb1-ad35-4719-8f2b-591fa5965bc5" (UID: "2f79cdb1-ad35-4719-8f2b-591fa5965bc5"). InnerVolumeSpecName "kube-api-access-xsc8d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:15:56.618504 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:56.618482    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istio-podinfo" pod "2f79cdb1-ad35-4719-8f2b-591fa5965bc5" (UID: "2f79cdb1-ad35-4719-8f2b-591fa5965bc5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 22:15:56.618504 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:56.618492    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istio-token" pod "2f79cdb1-ad35-4719-8f2b-591fa5965bc5" (UID: "2f79cdb1-ad35-4719-8f2b-591fa5965bc5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:15:56.618688 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:56.618657    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istio-envoy" pod "2f79cdb1-ad35-4719-8f2b-591fa5965bc5" (UID: "2f79cdb1-ad35-4719-8f2b-591fa5965bc5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:15:56.717472 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:56.717439    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xsc8d\" (UniqueName: \"kubernetes.io/projected/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-kube-api-access-xsc8d\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:56.717472 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:56.717469    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:56.717472 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:56.717479    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:56.717687 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:56.717486    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:56.717687 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:56.717493    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:56.717687 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:56.717501    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:56.717687 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:56.717510    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:56.717687 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:56.717524    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f79cdb1-ad35-4719-8f2b-591fa5965bc5-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:15:57.410700 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:57.410667    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs" event={"ID":"2f79cdb1-ad35-4719-8f2b-591fa5965bc5","Type":"ContainerDied","Data":"138204bcbffa8dcf5f0b79407b4e5e07f2dfb2d05093628ff2fff6312746a4fc"}
Oct 31 22:15:57.411375 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:57.410726    4460 scope.go:122] "RemoveContainer" containerID="d2c27a2e1df4c27f2a6f001cad7fe59f79c1ffcd718cef12018b0a61c4f2067d"
Oct 31 22:15:57.411375 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:57.410726    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"
Oct 31 22:15:57.471688 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:57.471654    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"]
Oct 31 22:15:57.485010 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:57.484980    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-710927-openshift-default-6b78cf7667-xqxjs"]
Oct 31 22:15:57.963489 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:57.963453    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:58.226573 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:58.226475    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2f79cdb1-ad35-4719-8f2b-591fa5965bc5" path="/var/lib/kubelet/pods/2f79cdb1-ad35-4719-8f2b-591fa5965bc5/volumes"
Oct 31 22:15:58.671207 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:58.671176    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"]
Oct 31 22:15:58.671759 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:58.671362    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp" podUID="2412d51b-ffbb-496d-a28e-1a1b3868e98a" containerName="istio-proxy" containerID="cri-o://0ccf07b9b9fc6878f75ee257a67a0c15e2fb9830b2a7e9a82ec91a542360eb62" gracePeriod=30
Oct 31 22:15:59.735343 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:59.735306    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:15:59.972870 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:59.972843    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-6b5dab-75bfb5c8df-nh5kr"
Oct 31 22:15:59.997663 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:15:59.997636    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-6b5dab-59c4c79ff9-n89dv"
Oct 31 22:16:00.061935 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.061895    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/15a6d325-3256-4d12-b698-fffafe8c39f0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15a6d325-3256-4d12-b698-fffafe8c39f0-plugins-config-volume\") pod \"15a6d325-3256-4d12-b698-fffafe8c39f0\" (UID: \"15a6d325-3256-4d12-b698-fffafe8c39f0\") "
Oct 31 22:16:00.062616 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.062591    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/15a6d325-3256-4d12-b698-fffafe8c39f0-kube-api-access-kpgpt\" (UniqueName: \"kubernetes.io/projected/15a6d325-3256-4d12-b698-fffafe8c39f0-kube-api-access-kpgpt\") pod \"15a6d325-3256-4d12-b698-fffafe8c39f0\" (UID: \"15a6d325-3256-4d12-b698-fffafe8c39f0\") "
Oct 31 22:16:00.062920 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.062897    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/afe24009-66f9-4972-9e31-709ec4fa2acb-kube-api-access-9hqxg\" (UniqueName: \"kubernetes.io/projected/afe24009-66f9-4972-9e31-709ec4fa2acb-kube-api-access-9hqxg\") pod \"afe24009-66f9-4972-9e31-709ec4fa2acb\" (UID: \"afe24009-66f9-4972-9e31-709ec4fa2acb\") "
Oct 31 22:16:00.063470 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.062465    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15a6d325-3256-4d12-b698-fffafe8c39f0-plugins-config-volume" pod "15a6d325-3256-4d12-b698-fffafe8c39f0" (UID: "15a6d325-3256-4d12-b698-fffafe8c39f0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:16:00.063658 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.063631    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/afe24009-66f9-4972-9e31-709ec4fa2acb-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afe24009-66f9-4972-9e31-709ec4fa2acb-plugins-config-volume\") pod \"afe24009-66f9-4972-9e31-709ec4fa2acb\" (UID: \"afe24009-66f9-4972-9e31-709ec4fa2acb\") "
Oct 31 22:16:00.064667 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.064203    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afe24009-66f9-4972-9e31-709ec4fa2acb-plugins-config-volume" pod "afe24009-66f9-4972-9e31-709ec4fa2acb" (UID: "afe24009-66f9-4972-9e31-709ec4fa2acb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:16:00.064667 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.064622    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afe24009-66f9-4972-9e31-709ec4fa2acb-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:00.064667 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.064645    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15a6d325-3256-4d12-b698-fffafe8c39f0-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:00.065775 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.065750    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15a6d325-3256-4d12-b698-fffafe8c39f0-kube-api-access-kpgpt" pod "15a6d325-3256-4d12-b698-fffafe8c39f0" (UID: "15a6d325-3256-4d12-b698-fffafe8c39f0"). InnerVolumeSpecName "kube-api-access-kpgpt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:16:00.066587 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.066564    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afe24009-66f9-4972-9e31-709ec4fa2acb-kube-api-access-9hqxg" pod "afe24009-66f9-4972-9e31-709ec4fa2acb" (UID: "afe24009-66f9-4972-9e31-709ec4fa2acb"). InnerVolumeSpecName "kube-api-access-9hqxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:16:00.166582 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.166483    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hqxg\" (UniqueName: \"kubernetes.io/projected/afe24009-66f9-4972-9e31-709ec4fa2acb-kube-api-access-9hqxg\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:00.166582 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.166519    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpgpt\" (UniqueName: \"kubernetes.io/projected/15a6d325-3256-4d12-b698-fffafe8c39f0-kube-api-access-kpgpt\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:00.435924 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.435839    4460 generic.go:356] "Generic (PLEG): container finished" podID="15a6d325-3256-4d12-b698-fffafe8c39f0" containerID="c826d9add18ef144496b83592c85332bf74d18da80d5972625ebb8ed5493ba9a" exitCode=0
Oct 31 22:16:00.435924 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.435887    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-6b5dab-59c4c79ff9-n89dv"
Oct 31 22:16:00.436182 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.435922    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-6b5dab-59c4c79ff9-n89dv" event={"ID":"15a6d325-3256-4d12-b698-fffafe8c39f0","Type":"ContainerDied","Data":"c826d9add18ef144496b83592c85332bf74d18da80d5972625ebb8ed5493ba9a"}
Oct 31 22:16:00.436182 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.435992    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-6b5dab-59c4c79ff9-n89dv" event={"ID":"15a6d325-3256-4d12-b698-fffafe8c39f0","Type":"ContainerDied","Data":"f112a00ddae28d1af7694ab081d91a25fae836136d44b35ef653b13e7e5c13f4"}
Oct 31 22:16:00.436182 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.436022    4460 scope.go:122] "RemoveContainer" containerID="c826d9add18ef144496b83592c85332bf74d18da80d5972625ebb8ed5493ba9a"
Oct 31 22:16:00.437696 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.437666    4460 generic.go:356] "Generic (PLEG): container finished" podID="afe24009-66f9-4972-9e31-709ec4fa2acb" containerID="ff31202536e63724b16d8793d4bee396ff047b09e5ba5cc72974944c7baa7c23" exitCode=0
Oct 31 22:16:00.437821 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.437702    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-6b5dab-75bfb5c8df-nh5kr"
Oct 31 22:16:00.437821 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.437742    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-6b5dab-75bfb5c8df-nh5kr" event={"ID":"afe24009-66f9-4972-9e31-709ec4fa2acb","Type":"ContainerDied","Data":"ff31202536e63724b16d8793d4bee396ff047b09e5ba5cc72974944c7baa7c23"}
Oct 31 22:16:00.437821 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.437792    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-6b5dab-75bfb5c8df-nh5kr" event={"ID":"afe24009-66f9-4972-9e31-709ec4fa2acb","Type":"ContainerDied","Data":"c05a5b0ed67c75c2e9e06e7b0785dd5a20c9a49918c3817d61b0e7a0e962e26b"}
Oct 31 22:16:00.447132 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.447114    4460 scope.go:122] "RemoveContainer" containerID="c826d9add18ef144496b83592c85332bf74d18da80d5972625ebb8ed5493ba9a"
Oct 31 22:16:00.447432 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:16:00.447414    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c826d9add18ef144496b83592c85332bf74d18da80d5972625ebb8ed5493ba9a\": container with ID starting with c826d9add18ef144496b83592c85332bf74d18da80d5972625ebb8ed5493ba9a not found: ID does not exist" containerID="c826d9add18ef144496b83592c85332bf74d18da80d5972625ebb8ed5493ba9a"
Oct 31 22:16:00.447491 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.447439    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c826d9add18ef144496b83592c85332bf74d18da80d5972625ebb8ed5493ba9a"} err="failed to get container status \"c826d9add18ef144496b83592c85332bf74d18da80d5972625ebb8ed5493ba9a\": rpc error: code = NotFound desc = could not find container \"c826d9add18ef144496b83592c85332bf74d18da80d5972625ebb8ed5493ba9a\": container with ID starting with c826d9add18ef144496b83592c85332bf74d18da80d5972625ebb8ed5493ba9a not found: ID does not exist"
Oct 31 22:16:00.447491 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.447456    4460 scope.go:122] "RemoveContainer" containerID="ff31202536e63724b16d8793d4bee396ff047b09e5ba5cc72974944c7baa7c23"
Oct 31 22:16:00.457524 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.457504    4460 scope.go:122] "RemoveContainer" containerID="ff31202536e63724b16d8793d4bee396ff047b09e5ba5cc72974944c7baa7c23"
Oct 31 22:16:00.457823 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:16:00.457804    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff31202536e63724b16d8793d4bee396ff047b09e5ba5cc72974944c7baa7c23\": container with ID starting with ff31202536e63724b16d8793d4bee396ff047b09e5ba5cc72974944c7baa7c23 not found: ID does not exist" containerID="ff31202536e63724b16d8793d4bee396ff047b09e5ba5cc72974944c7baa7c23"
Oct 31 22:16:00.457885 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.457831    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff31202536e63724b16d8793d4bee396ff047b09e5ba5cc72974944c7baa7c23"} err="failed to get container status \"ff31202536e63724b16d8793d4bee396ff047b09e5ba5cc72974944c7baa7c23\": rpc error: code = NotFound desc = could not find container \"ff31202536e63724b16d8793d4bee396ff047b09e5ba5cc72974944c7baa7c23\": container with ID starting with ff31202536e63724b16d8793d4bee396ff047b09e5ba5cc72974944c7baa7c23 not found: ID does not exist"
Oct 31 22:16:00.494423 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.494397    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-6b5dab-59c4c79ff9-n89dv"]
Oct 31 22:16:00.509266 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.509240    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-6b5dab-59c4c79ff9-n89dv"]
Oct 31 22:16:00.538147 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.538125    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-6b5dab-75bfb5c8df-nh5kr"]
Oct 31 22:16:00.545030 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.545006    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-6b5dab-75bfb5c8df-nh5kr"]
Oct 31 22:16:00.911213 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.911168    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-6b5dab-59c4c79ff9-n89dv" podUID="15a6d325-3256-4d12-b698-fffafe8c39f0" containerName="payload-pre-processing" probeResult="failure" output="timeout: failed to connect service \"10.133.0.146:50052\" within 1s: context deadline exceeded"
Oct 31 22:16:00.911674 ip-10-0-136-157 kubenswrapper[4460]: W1031 22:16:00.911257    4460 logging.go:55] [core] [Channel #15717 SubChannel #15718]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.146:50052", ServerName: "10.133.0.146:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.146:50052: operation was canceled"
Oct 31 22:16:00.911674 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.911627    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-6b5dab-75bfb5c8df-nh5kr" podUID="afe24009-66f9-4972-9e31-709ec4fa2acb" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.133.0.145:50052\" within 1s: context deadline exceeded"
Oct 31 22:16:00.911809 ip-10-0-136-157 kubenswrapper[4460]: W1031 22:16:00.911723    4460 logging.go:55] [core] [Channel #15719 SubChannel #15720]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.145:50052", ServerName: "10.133.0.145:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.145:50052: operation was canceled"
Oct 31 22:16:00.921419 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.921397    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ssl9_26069598-5dc4-4c96-b55f-dd2f09884419/ovn-acl-logging/0.log"
Oct 31 22:16:00.931251 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:00.931234    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ssl9_26069598-5dc4-4c96-b55f-dd2f09884419/ovn-acl-logging/0.log"
Oct 31 22:16:01.497874 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:01.497839    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 22:16:01.521197 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:01.521165    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:02.227745 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:02.227696    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="15a6d325-3256-4d12-b698-fffafe8c39f0" path="/var/lib/kubelet/pods/15a6d325-3256-4d12-b698-fffafe8c39f0/volumes"
Oct 31 22:16:02.228193 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:02.228176    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="afe24009-66f9-4972-9e31-709ec4fa2acb" path="/var/lib/kubelet/pods/afe24009-66f9-4972-9e31-709ec4fa2acb/volumes"
Oct 31 22:16:02.375607 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:02.375566    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:02.494626 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:02.494474    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:03.341385 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:03.341347    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:03.952148 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:03.952085    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:16:04.034332 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.034297    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istiod-ca-cert\") pod \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") "
Oct 31 22:16:04.034534 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.034389    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istio-envoy\") pod \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") "
Oct 31 22:16:04.034534 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.034459    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2412d51b-ffbb-496d-a28e-1a1b3868e98a-kube-api-access-hd9zj\" (UniqueName: \"kubernetes.io/projected/2412d51b-ffbb-496d-a28e-1a1b3868e98a-kube-api-access-hd9zj\") pod \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") "
Oct 31 22:16:04.034661 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.034539    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-workload-certs\") pod \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") "
Oct 31 22:16:04.034661 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.034579    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istio-token\" (UniqueName: \"kubernetes.io/projected/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istio-token\") pod \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") "
Oct 31 22:16:04.034786 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.034678    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istio-podinfo\") pod \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") "
Oct 31 22:16:04.034786 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.034743    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-credential-socket\") pod \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") "
Oct 31 22:16:04.034910 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.034792    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-workload-socket\") pod \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") "
Oct 31 22:16:04.035122 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.035091    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-workload-certs" pod "2412d51b-ffbb-496d-a28e-1a1b3868e98a" (UID: "2412d51b-ffbb-496d-a28e-1a1b3868e98a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:16:04.035194 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.035111    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istiod-ca-cert" pod "2412d51b-ffbb-496d-a28e-1a1b3868e98a" (UID: "2412d51b-ffbb-496d-a28e-1a1b3868e98a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:16:04.035432 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.035390    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-credential-socket" pod "2412d51b-ffbb-496d-a28e-1a1b3868e98a" (UID: "2412d51b-ffbb-496d-a28e-1a1b3868e98a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:16:04.035432 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.035419    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-workload-socket" pod "2412d51b-ffbb-496d-a28e-1a1b3868e98a" (UID: "2412d51b-ffbb-496d-a28e-1a1b3868e98a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:16:04.035803 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.035783    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istio-data\") pod \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\" (UID: \"2412d51b-ffbb-496d-a28e-1a1b3868e98a\") "
Oct 31 22:16:04.036482 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.036151    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istio-data" pod "2412d51b-ffbb-496d-a28e-1a1b3868e98a" (UID: "2412d51b-ffbb-496d-a28e-1a1b3868e98a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:16:04.037108 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.037086    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:04.037259 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.037244    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:04.037391 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.037372    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:04.037503 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.037393    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:04.037503 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.037418    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:04.037503 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.037191    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istio-envoy" pod "2412d51b-ffbb-496d-a28e-1a1b3868e98a" (UID: "2412d51b-ffbb-496d-a28e-1a1b3868e98a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:16:04.037503 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.037366    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istio-podinfo" pod "2412d51b-ffbb-496d-a28e-1a1b3868e98a" (UID: "2412d51b-ffbb-496d-a28e-1a1b3868e98a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 22:16:04.037748 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.037728    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2412d51b-ffbb-496d-a28e-1a1b3868e98a-kube-api-access-hd9zj" pod "2412d51b-ffbb-496d-a28e-1a1b3868e98a" (UID: "2412d51b-ffbb-496d-a28e-1a1b3868e98a"). InnerVolumeSpecName "kube-api-access-hd9zj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:16:04.037847 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.037822    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istio-token" pod "2412d51b-ffbb-496d-a28e-1a1b3868e98a" (UID: "2412d51b-ffbb-496d-a28e-1a1b3868e98a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:16:04.139276 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.139243    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:04.139276 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.139274    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hd9zj\" (UniqueName: \"kubernetes.io/projected/2412d51b-ffbb-496d-a28e-1a1b3868e98a-kube-api-access-hd9zj\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:04.139276 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.139286    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:04.139529 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.139294    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2412d51b-ffbb-496d-a28e-1a1b3868e98a-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:04.472073 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.472038    4460 generic.go:356] "Generic (PLEG): container finished" podID="2412d51b-ffbb-496d-a28e-1a1b3868e98a" containerID="0ccf07b9b9fc6878f75ee257a67a0c15e2fb9830b2a7e9a82ec91a542360eb62" exitCode=0
Oct 31 22:16:04.472513 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.472094    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"
Oct 31 22:16:04.472513 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.472123    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp" event={"ID":"2412d51b-ffbb-496d-a28e-1a1b3868e98a","Type":"ContainerDied","Data":"0ccf07b9b9fc6878f75ee257a67a0c15e2fb9830b2a7e9a82ec91a542360eb62"}
Oct 31 22:16:04.472513 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.472178    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp" event={"ID":"2412d51b-ffbb-496d-a28e-1a1b3868e98a","Type":"ContainerDied","Data":"51f497beb8a5fb4bf775e23cd589d0148b45483d00ad7ad813259be32a91a295"}
Oct 31 22:16:04.472513 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.472200    4460 scope.go:122] "RemoveContainer" containerID="0ccf07b9b9fc6878f75ee257a67a0c15e2fb9830b2a7e9a82ec91a542360eb62"
Oct 31 22:16:04.483170 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.483152    4460 scope.go:122] "RemoveContainer" containerID="0ccf07b9b9fc6878f75ee257a67a0c15e2fb9830b2a7e9a82ec91a542360eb62"
Oct 31 22:16:04.483441 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:16:04.483420    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ccf07b9b9fc6878f75ee257a67a0c15e2fb9830b2a7e9a82ec91a542360eb62\": container with ID starting with 0ccf07b9b9fc6878f75ee257a67a0c15e2fb9830b2a7e9a82ec91a542360eb62 not found: ID does not exist" containerID="0ccf07b9b9fc6878f75ee257a67a0c15e2fb9830b2a7e9a82ec91a542360eb62"
Oct 31 22:16:04.483495 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.483450    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ccf07b9b9fc6878f75ee257a67a0c15e2fb9830b2a7e9a82ec91a542360eb62"} err="failed to get container status \"0ccf07b9b9fc6878f75ee257a67a0c15e2fb9830b2a7e9a82ec91a542360eb62\": rpc error: code = NotFound desc = could not find container \"0ccf07b9b9fc6878f75ee257a67a0c15e2fb9830b2a7e9a82ec91a542360eb62\": container with ID starting with 0ccf07b9b9fc6878f75ee257a67a0c15e2fb9830b2a7e9a82ec91a542360eb62 not found: ID does not exist"
Oct 31 22:16:04.533998 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.533970    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"]
Oct 31 22:16:04.547994 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.547973    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-vnvpp"]
Oct 31 22:16:04.814102 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:04.814015    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:06.012888 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:06.012852    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:06.226293 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:06.226256    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2412d51b-ffbb-496d-a28e-1a1b3868e98a" path="/var/lib/kubelet/pods/2412d51b-ffbb-496d-a28e-1a1b3868e98a/volumes"
Oct 31 22:16:07.484718 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:07.484682    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:09.253881 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:09.253833    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:11.090969 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:11.088357    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:12.505728 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:12.505691    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:12.617177 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:12.617142    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:12.834901 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:12.834863    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:12.853226 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:12.853190    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"]
Oct 31 22:16:12.853978 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:12.853445    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw" podUID="581aed43-b880-434d-a7ca-3d0c78e54c38" containerName="istio-proxy" containerID="cri-o://03436e419060bbbf2273b87c3eddb77ea8b46003dc00bd65b0e678525bcb7ad7" gracePeriod=30
Oct 31 22:16:12.976151 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:12.976099    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:14.665127 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:14.665087    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:16.412226 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:16.412193    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:18.122493 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.122427    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:18.138219 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.138192    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 22:16:18.262970 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.261800    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-istio-envoy\") pod \"581aed43-b880-434d-a7ca-3d0c78e54c38\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") "
Oct 31 22:16:18.262970 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.261921    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/581aed43-b880-434d-a7ca-3d0c78e54c38-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/581aed43-b880-434d-a7ca-3d0c78e54c38-istio-podinfo\") pod \"581aed43-b880-434d-a7ca-3d0c78e54c38\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") "
Oct 31 22:16:18.262970 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.262073    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/581aed43-b880-434d-a7ca-3d0c78e54c38-kube-api-access-bd6dl\" (UniqueName: \"kubernetes.io/projected/581aed43-b880-434d-a7ca-3d0c78e54c38-kube-api-access-bd6dl\") pod \"581aed43-b880-434d-a7ca-3d0c78e54c38\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") "
Oct 31 22:16:18.262970 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.262107    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/581aed43-b880-434d-a7ca-3d0c78e54c38-istio-token\" (UniqueName: \"kubernetes.io/projected/581aed43-b880-434d-a7ca-3d0c78e54c38-istio-token\") pod \"581aed43-b880-434d-a7ca-3d0c78e54c38\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") "
Oct 31 22:16:18.262970 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.262176    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-workload-certs\") pod \"581aed43-b880-434d-a7ca-3d0c78e54c38\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") "
Oct 31 22:16:18.262970 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.262215    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-workload-socket\") pod \"581aed43-b880-434d-a7ca-3d0c78e54c38\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") "
Oct 31 22:16:18.262970 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.262268    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/581aed43-b880-434d-a7ca-3d0c78e54c38-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/581aed43-b880-434d-a7ca-3d0c78e54c38-istiod-ca-cert\") pod \"581aed43-b880-434d-a7ca-3d0c78e54c38\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") "
Oct 31 22:16:18.262970 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.262345    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-credential-socket\") pod \"581aed43-b880-434d-a7ca-3d0c78e54c38\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") "
Oct 31 22:16:18.262970 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.262420    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-istio-data\") pod \"581aed43-b880-434d-a7ca-3d0c78e54c38\" (UID: \"581aed43-b880-434d-a7ca-3d0c78e54c38\") "
Oct 31 22:16:18.264819 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.263796    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-workload-certs" pod "581aed43-b880-434d-a7ca-3d0c78e54c38" (UID: "581aed43-b880-434d-a7ca-3d0c78e54c38"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:16:18.264819 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.264030    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-workload-socket" pod "581aed43-b880-434d-a7ca-3d0c78e54c38" (UID: "581aed43-b880-434d-a7ca-3d0c78e54c38"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:16:18.264819 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.264292    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/581aed43-b880-434d-a7ca-3d0c78e54c38-istiod-ca-cert" pod "581aed43-b880-434d-a7ca-3d0c78e54c38" (UID: "581aed43-b880-434d-a7ca-3d0c78e54c38"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:16:18.264819 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.264575    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-credential-socket" pod "581aed43-b880-434d-a7ca-3d0c78e54c38" (UID: "581aed43-b880-434d-a7ca-3d0c78e54c38"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:16:18.335186 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.324745    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:18.335186 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.324780    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:18.335186 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.324794    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/581aed43-b880-434d-a7ca-3d0c78e54c38-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:18.335186 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.324807    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:18.351131 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.351067    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-istio-data" pod "581aed43-b880-434d-a7ca-3d0c78e54c38" (UID: "581aed43-b880-434d-a7ca-3d0c78e54c38"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:16:18.378236 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.378130    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-istio-envoy" pod "581aed43-b880-434d-a7ca-3d0c78e54c38" (UID: "581aed43-b880-434d-a7ca-3d0c78e54c38"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:16:18.378236 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.378205    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/581aed43-b880-434d-a7ca-3d0c78e54c38-istio-token" pod "581aed43-b880-434d-a7ca-3d0c78e54c38" (UID: "581aed43-b880-434d-a7ca-3d0c78e54c38"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:16:18.378476 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.378256    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/581aed43-b880-434d-a7ca-3d0c78e54c38-kube-api-access-bd6dl" pod "581aed43-b880-434d-a7ca-3d0c78e54c38" (UID: "581aed43-b880-434d-a7ca-3d0c78e54c38"). InnerVolumeSpecName "kube-api-access-bd6dl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:16:18.383062 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.382598    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/581aed43-b880-434d-a7ca-3d0c78e54c38-istio-podinfo" pod "581aed43-b880-434d-a7ca-3d0c78e54c38" (UID: "581aed43-b880-434d-a7ca-3d0c78e54c38"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 22:16:18.426739 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.426708    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:18.426739 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.426735    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/581aed43-b880-434d-a7ca-3d0c78e54c38-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:18.426739 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.426746    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/581aed43-b880-434d-a7ca-3d0c78e54c38-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:18.426982 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.426756    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bd6dl\" (UniqueName: \"kubernetes.io/projected/581aed43-b880-434d-a7ca-3d0c78e54c38-kube-api-access-bd6dl\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:18.426982 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.426765    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/581aed43-b880-434d-a7ca-3d0c78e54c38-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:18.582807 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.582771    4460 generic.go:356] "Generic (PLEG): container finished" podID="581aed43-b880-434d-a7ca-3d0c78e54c38" containerID="03436e419060bbbf2273b87c3eddb77ea8b46003dc00bd65b0e678525bcb7ad7" exitCode=0
Oct 31 22:16:18.583009 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.582822    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"
Oct 31 22:16:18.583009 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.582825    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw" event={"ID":"581aed43-b880-434d-a7ca-3d0c78e54c38","Type":"ContainerDied","Data":"03436e419060bbbf2273b87c3eddb77ea8b46003dc00bd65b0e678525bcb7ad7"}
Oct 31 22:16:18.583009 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.582990    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw" event={"ID":"581aed43-b880-434d-a7ca-3d0c78e54c38","Type":"ContainerDied","Data":"a6d6e20b3adb71576f3dc2a25a0de81c44b6847f0d48e69204666c2f12b6a0bd"}
Oct 31 22:16:18.583225 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.583029    4460 scope.go:122] "RemoveContainer" containerID="03436e419060bbbf2273b87c3eddb77ea8b46003dc00bd65b0e678525bcb7ad7"
Oct 31 22:16:18.595933 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.595913    4460 scope.go:122] "RemoveContainer" containerID="2eb8df3c9a4d36a0a64881d84caeb3cde2c7883b7f41698a162f7c0fa2083a54"
Oct 31 22:16:18.605714 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.605679    4460 scope.go:122] "RemoveContainer" containerID="03436e419060bbbf2273b87c3eddb77ea8b46003dc00bd65b0e678525bcb7ad7"
Oct 31 22:16:18.606025 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:16:18.606007    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03436e419060bbbf2273b87c3eddb77ea8b46003dc00bd65b0e678525bcb7ad7\": container with ID starting with 03436e419060bbbf2273b87c3eddb77ea8b46003dc00bd65b0e678525bcb7ad7 not found: ID does not exist" containerID="03436e419060bbbf2273b87c3eddb77ea8b46003dc00bd65b0e678525bcb7ad7"
Oct 31 22:16:18.606121 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.606033    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03436e419060bbbf2273b87c3eddb77ea8b46003dc00bd65b0e678525bcb7ad7"} err="failed to get container status \"03436e419060bbbf2273b87c3eddb77ea8b46003dc00bd65b0e678525bcb7ad7\": rpc error: code = NotFound desc = could not find container \"03436e419060bbbf2273b87c3eddb77ea8b46003dc00bd65b0e678525bcb7ad7\": container with ID starting with 03436e419060bbbf2273b87c3eddb77ea8b46003dc00bd65b0e678525bcb7ad7 not found: ID does not exist"
Oct 31 22:16:18.606121 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.606049    4460 scope.go:122] "RemoveContainer" containerID="2eb8df3c9a4d36a0a64881d84caeb3cde2c7883b7f41698a162f7c0fa2083a54"
Oct 31 22:16:18.606280 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:16:18.606264    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2eb8df3c9a4d36a0a64881d84caeb3cde2c7883b7f41698a162f7c0fa2083a54\": container with ID starting with 2eb8df3c9a4d36a0a64881d84caeb3cde2c7883b7f41698a162f7c0fa2083a54 not found: ID does not exist" containerID="2eb8df3c9a4d36a0a64881d84caeb3cde2c7883b7f41698a162f7c0fa2083a54"
Oct 31 22:16:18.606328 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.606283    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eb8df3c9a4d36a0a64881d84caeb3cde2c7883b7f41698a162f7c0fa2083a54"} err="failed to get container status \"2eb8df3c9a4d36a0a64881d84caeb3cde2c7883b7f41698a162f7c0fa2083a54\": rpc error: code = NotFound desc = could not find container \"2eb8df3c9a4d36a0a64881d84caeb3cde2c7883b7f41698a162f7c0fa2083a54\": container with ID starting with 2eb8df3c9a4d36a0a64881d84caeb3cde2c7883b7f41698a162f7c0fa2083a54 not found: ID does not exist"
Oct 31 22:16:18.643552 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.643485    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"]
Oct 31 22:16:18.656746 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:18.656716    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-6b5dab-openshift-default-684774b79f-sbrtw"]
Oct 31 22:16:19.907644 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:19.907609    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:20.226796 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:20.226718    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="581aed43-b880-434d-a7ca-3d0c78e54c38" path="/var/lib/kubelet/pods/581aed43-b880-434d-a7ca-3d0c78e54c38/volumes"
Oct 31 22:16:21.452924 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:16:21.452887    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:16:21.765301 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:21.765258    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:23.531481 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:23.531426    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:24.067777 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:24.067743    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:24.209024 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:24.208985    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:25.330468 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:25.330432    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:27.000398 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:27.000363    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:28.525994 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:28.525926    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:28.800294 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:28.799930    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"]
Oct 31 22:16:28.800817 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:28.800767    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd" podUID="2ce9ce6d-57d6-4085-94e5-2f7f600b81b3" containerName="istio-proxy" containerID="cri-o://15150ad7d6becc4804f2bb1576e4065237440c5ffa1e8c7580109aa49f7289f9" gracePeriod=30
Oct 31 22:16:29.077028 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:29.076180    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:29.204287 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:29.204244    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:29.336056 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:29.335818    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:29.683877 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:29.683791    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:31.043119 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:31.043078    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:32.742788 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:32.742726    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:34.096971 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.096900    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:16:34.217025 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.216975    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istiod-ca-cert\") pod \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") "
Oct 31 22:16:34.217226 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.217039    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-credential-socket\") pod \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") "
Oct 31 22:16:34.217226 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.217079    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-workload-socket\") pod \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") "
Oct 31 22:16:34.217226 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.217170    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istio-envoy\") pod \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") "
Oct 31 22:16:34.217729 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.217308    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istio-podinfo\") pod \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") "
Oct 31 22:16:34.217729 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.217391    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-workload-certs\") pod \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") "
Oct 31 22:16:34.217729 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.217437    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istio-token\" (UniqueName: \"kubernetes.io/projected/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istio-token\") pod \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") "
Oct 31 22:16:34.217729 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.217500    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-kube-api-access-lwlg7\" (UniqueName: \"kubernetes.io/projected/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-kube-api-access-lwlg7\") pod \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") "
Oct 31 22:16:34.217729 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.217562    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istio-data\") pod \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\" (UID: \"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3\") "
Oct 31 22:16:34.218386 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.218330    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-credential-socket" pod "2ce9ce6d-57d6-4085-94e5-2f7f600b81b3" (UID: "2ce9ce6d-57d6-4085-94e5-2f7f600b81b3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:16:34.218716 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.218562    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-workload-certs" pod "2ce9ce6d-57d6-4085-94e5-2f7f600b81b3" (UID: "2ce9ce6d-57d6-4085-94e5-2f7f600b81b3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:16:34.218716 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.218582    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istiod-ca-cert" pod "2ce9ce6d-57d6-4085-94e5-2f7f600b81b3" (UID: "2ce9ce6d-57d6-4085-94e5-2f7f600b81b3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:16:34.219237 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.219208    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-workload-socket" pod "2ce9ce6d-57d6-4085-94e5-2f7f600b81b3" (UID: "2ce9ce6d-57d6-4085-94e5-2f7f600b81b3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:16:34.219352 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.219278    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istio-data" pod "2ce9ce6d-57d6-4085-94e5-2f7f600b81b3" (UID: "2ce9ce6d-57d6-4085-94e5-2f7f600b81b3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:16:34.221685 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.221633    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istio-envoy" pod "2ce9ce6d-57d6-4085-94e5-2f7f600b81b3" (UID: "2ce9ce6d-57d6-4085-94e5-2f7f600b81b3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:16:34.222273 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.222235    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istio-podinfo" pod "2ce9ce6d-57d6-4085-94e5-2f7f600b81b3" (UID: "2ce9ce6d-57d6-4085-94e5-2f7f600b81b3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 22:16:34.222382 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.222323    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istio-token" pod "2ce9ce6d-57d6-4085-94e5-2f7f600b81b3" (UID: "2ce9ce6d-57d6-4085-94e5-2f7f600b81b3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:16:34.224784 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.224759    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-kube-api-access-lwlg7" pod "2ce9ce6d-57d6-4085-94e5-2f7f600b81b3" (UID: "2ce9ce6d-57d6-4085-94e5-2f7f600b81b3"). InnerVolumeSpecName "kube-api-access-lwlg7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:16:34.319201 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.319168    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:34.319201 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.319197    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:34.319374 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.319206    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwlg7\" (UniqueName: \"kubernetes.io/projected/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-kube-api-access-lwlg7\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:34.319374 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.319219    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:34.319374 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.319228    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:34.319374 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.319235    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:34.319374 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.319244    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:34.319374 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.319251    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:34.319374 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.319259    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:34.621119 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.621085    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:34.731434 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.731394    4460 generic.go:356] "Generic (PLEG): container finished" podID="2ce9ce6d-57d6-4085-94e5-2f7f600b81b3" containerID="15150ad7d6becc4804f2bb1576e4065237440c5ffa1e8c7580109aa49f7289f9" exitCode=0
Oct 31 22:16:34.731629 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.731442    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"
Oct 31 22:16:34.731629 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.731473    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd" event={"ID":"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3","Type":"ContainerDied","Data":"15150ad7d6becc4804f2bb1576e4065237440c5ffa1e8c7580109aa49f7289f9"}
Oct 31 22:16:34.731629 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.731534    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd" event={"ID":"2ce9ce6d-57d6-4085-94e5-2f7f600b81b3","Type":"ContainerDied","Data":"dcdbbd32218b06e2de6fbb2e342350a90d681c3b8c0ac66de81171f013ca888d"}
Oct 31 22:16:34.731629 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.731563    4460 scope.go:122] "RemoveContainer" containerID="15150ad7d6becc4804f2bb1576e4065237440c5ffa1e8c7580109aa49f7289f9"
Oct 31 22:16:34.742475 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.742454    4460 scope.go:122] "RemoveContainer" containerID="15150ad7d6becc4804f2bb1576e4065237440c5ffa1e8c7580109aa49f7289f9"
Oct 31 22:16:34.742765 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:16:34.742744    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15150ad7d6becc4804f2bb1576e4065237440c5ffa1e8c7580109aa49f7289f9\": container with ID starting with 15150ad7d6becc4804f2bb1576e4065237440c5ffa1e8c7580109aa49f7289f9 not found: ID does not exist" containerID="15150ad7d6becc4804f2bb1576e4065237440c5ffa1e8c7580109aa49f7289f9"
Oct 31 22:16:34.742822 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.742773    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15150ad7d6becc4804f2bb1576e4065237440c5ffa1e8c7580109aa49f7289f9"} err="failed to get container status \"15150ad7d6becc4804f2bb1576e4065237440c5ffa1e8c7580109aa49f7289f9\": rpc error: code = NotFound desc = could not find container \"15150ad7d6becc4804f2bb1576e4065237440c5ffa1e8c7580109aa49f7289f9\": container with ID starting with 15150ad7d6becc4804f2bb1576e4065237440c5ffa1e8c7580109aa49f7289f9 not found: ID does not exist"
Oct 31 22:16:34.790542 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.790511    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"]
Oct 31 22:16:34.803737 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:34.803711    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-96tvd"]
Oct 31 22:16:36.226236 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:36.226206    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2ce9ce6d-57d6-4085-94e5-2f7f600b81b3" path="/var/lib/kubelet/pods/2ce9ce6d-57d6-4085-94e5-2f7f600b81b3/volumes"
Oct 31 22:16:36.304850 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:36.304813    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:36.424609 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:36.424560    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:36.533980 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:36.533915    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:36.617136 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:36.617097    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"]
Oct 31 22:16:36.617412 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:36.617379    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6" podUID="876bda78-8ad9-445d-baa7-37c9ab1b4237" containerName="istio-proxy" containerID="cri-o://b43dcbff2f72f38b213b9d4a60464b0ac80c5ff97bad7c2b7bebc47ad4b08641" gracePeriod=30
Oct 31 22:16:36.643539 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:36.643496    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:38.334916 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:38.334865    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:38.567317 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:38.567279    4460 patch_prober.go:29] interesting pod/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 22:16:38.567494 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:38.567367    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6" podUID="876bda78-8ad9-445d-baa7-37c9ab1b4237" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 22:16:40.110192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:40.110154    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:41.823427 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:41.823184    4460 generic.go:356] "Generic (PLEG): container finished" podID="876bda78-8ad9-445d-baa7-37c9ab1b4237" containerID="b43dcbff2f72f38b213b9d4a60464b0ac80c5ff97bad7c2b7bebc47ad4b08641" exitCode=0
Oct 31 22:16:41.823427 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:41.823282    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6" event={"ID":"876bda78-8ad9-445d-baa7-37c9ab1b4237","Type":"ContainerDied","Data":"b43dcbff2f72f38b213b9d4a60464b0ac80c5ff97bad7c2b7bebc47ad4b08641"}
Oct 31 22:16:41.823427 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:41.823337    4460 scope.go:122] "RemoveContainer" containerID="7acdecfc197a314529514a6f056144366fbe170877bb9087af1d0967d3bffa27"
Oct 31 22:16:41.853662 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:41.853628    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:41.940116 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:41.940085    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:16:42.034740 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:42.034703    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/876bda78-8ad9-445d-baa7-37c9ab1b4237-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/876bda78-8ad9-445d-baa7-37c9ab1b4237-istiod-ca-cert\") pod \"876bda78-8ad9-445d-baa7-37c9ab1b4237\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") "
Oct 31 22:16:42.034948 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:42.034759    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-istio-data\") pod \"876bda78-8ad9-445d-baa7-37c9ab1b4237\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") "
Oct 31 22:16:42.034948 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:42.034796    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-credential-socket\") pod \"876bda78-8ad9-445d-baa7-37c9ab1b4237\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") "
Oct 31 22:16:42.034948 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:42.034869    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/876bda78-8ad9-445d-baa7-37c9ab1b4237-kube-api-access-29ccp\" (UniqueName: \"kubernetes.io/projected/876bda78-8ad9-445d-baa7-37c9ab1b4237-kube-api-access-29ccp\") pod \"876bda78-8ad9-445d-baa7-37c9ab1b4237\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") "
Oct 31 22:16:42.034948 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:42.034897    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-workload-socket\") pod \"876bda78-8ad9-445d-baa7-37c9ab1b4237\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") "
Oct 31 22:16:42.035150 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:42.034975    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-workload-certs\") pod \"876bda78-8ad9-445d-baa7-37c9ab1b4237\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") "
Oct 31 22:16:42.035150 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:42.035116    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/876bda78-8ad9-445d-baa7-37c9ab1b4237-istio-token\" (UniqueName: \"kubernetes.io/projected/876bda78-8ad9-445d-baa7-37c9ab1b4237-istio-token\") pod \"876bda78-8ad9-445d-baa7-37c9ab1b4237\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") "
Oct 31 22:16:42.035243 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:42.035183    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-istio-data" pod "876bda78-8ad9-445d-baa7-37c9ab1b4237" (UID: "876bda78-8ad9-445d-baa7-37c9ab1b4237"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:16:42.035243 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:42.035212    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/876bda78-8ad9-445d-baa7-37c9ab1b4237-istiod-ca-cert" pod "876bda78-8ad9-445d-baa7-37c9ab1b4237" (UID: "876bda78-8ad9-445d-baa7-37c9ab1b4237"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:16:42.035243 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:42.035219    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/876bda78-8ad9-445d-baa7-37c9ab1b4237-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/876bda78-8ad9-445d-baa7-37c9ab1b4237-istio-podinfo\") pod \"876bda78-8ad9-445d-baa7-37c9ab1b4237\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") "
Oct 31 22:16:42.035410 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:42.035286    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-istio-envoy\") pod \"876bda78-8ad9-445d-baa7-37c9ab1b4237\" (UID: \"876bda78-8ad9-445d-baa7-37c9ab1b4237\") "
Oct 31 22:16:42.036142 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:42.036118    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/876bda78-8ad9-445d-baa7-37c9ab1b4237-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:42.036317 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:42.036301    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:42.036460 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:42.036184    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-workload-certs" pod "876bda78-8ad9-445d-baa7-37c9ab1b4237" (UID: "876bda78-8ad9-445d-baa7-37c9ab1b4237"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:16:42.036460 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:42.036197    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-workload-socket" pod "876bda78-8ad9-445d-baa7-37c9ab1b4237" (UID: "876bda78-8ad9-445d-baa7-37c9ab1b4237"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:16:42.036674 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:42.036317    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-credential-socket" pod "876bda78-8ad9-445d-baa7-37c9ab1b4237" (UID: "876bda78-8ad9-445d-baa7-37c9ab1b4237"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:16:42.038592 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:42.038564    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/876bda78-8ad9-445d-baa7-37c9ab1b4237-kube-api-access-29ccp" pod "876bda78-8ad9-445d-baa7-37c9ab1b4237" (UID: "876bda78-8ad9-445d-baa7-37c9ab1b4237"). InnerVolumeSpecName "kube-api-access-29ccp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:16:42.038690 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:42.038617    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/876bda78-8ad9-445d-baa7-37c9ab1b4237-istio-token" pod "876bda78-8ad9-445d-baa7-37c9ab1b4237" (UID: "876bda78-8ad9-445d-baa7-37c9ab1b4237"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:16:42.038803 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:42.038779    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-istio-envoy" pod "876bda78-8ad9-445d-baa7-37c9ab1b4237" (UID: "876bda78-8ad9-445d-baa7-37c9ab1b4237"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:16:42.039260 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:42.039233    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/876bda78-8ad9-445d-baa7-37c9ab1b4237-istio-podinfo" pod "876bda78-8ad9-445d-baa7-37c9ab1b4237" (UID: "876bda78-8ad9-445d-baa7-37c9ab1b4237"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 22:16:42.137421 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:42.137320    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:42.137421 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:42.137356    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29ccp\" (UniqueName: \"kubernetes.io/projected/876bda78-8ad9-445d-baa7-37c9ab1b4237-kube-api-access-29ccp\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:42.137421 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:42.137370    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:42.137421 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:42.137382    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:42.137421 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:42.137393    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/876bda78-8ad9-445d-baa7-37c9ab1b4237-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:42.137421 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:42.137404    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/876bda78-8ad9-445d-baa7-37c9ab1b4237-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:42.137421 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:42.137416    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/876bda78-8ad9-445d-baa7-37c9ab1b4237-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:16:42.831176 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:42.831102    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"
Oct 31 22:16:42.831176 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:42.831130    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6" event={"ID":"876bda78-8ad9-445d-baa7-37c9ab1b4237","Type":"ContainerDied","Data":"0630a84895a050dad98b56e43b66e0850f17dc448357f7eb3d62964e3f993a78"}
Oct 31 22:16:42.831681 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:42.831224    4460 scope.go:122] "RemoveContainer" containerID="b43dcbff2f72f38b213b9d4a60464b0ac80c5ff97bad7c2b7bebc47ad4b08641"
Oct 31 22:16:42.903236 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:42.903195    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"]
Oct 31 22:16:42.916855 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:42.916820    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-c2d1b8-openshift-default-584f96b4df-fslk6"]
Oct 31 22:16:43.620375 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:43.620341    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:44.229444 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:44.229409    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="876bda78-8ad9-445d-baa7-37c9ab1b4237" path="/var/lib/kubelet/pods/876bda78-8ad9-445d-baa7-37c9ab1b4237/volumes"
Oct 31 22:16:44.240841 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:44.240805    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:45.407014 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:45.406979    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:47.134507 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:47.134463    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:47.654814 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:47.654771    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-4bd51b-7d76c5559f-r978k"]
Oct 31 22:16:47.690135 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:47.690100    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-4bd51b-6b775cddf9-ld82r"]
Oct 31 22:16:47.796648 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:47.796610    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:47.930470 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:47.930363    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:48.943210 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:48.943169    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:49.720113 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:49.720073    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 22:16:50.739737 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:50.739702    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:51.151850 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:51.151812    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:51.453236 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:16:51.453148    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:16:52.505387 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:52.505332    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:52.656114 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:52.656072    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-4bd51b-7d76c5559f-r978k" podUID="8a302e86-54e3-4d9f-81ee-350498a9149e" containerName="payload-processing" containerID="cri-o://d0d6a4d80081d692669527d2888b8d6d3d7a32a0b62eed694d1c8e1729c94cd6" gracePeriod=25
Oct 31 22:16:52.691206 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:52.691165    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-4bd51b-6b775cddf9-ld82r" podUID="d141422b-120e-4dc4-bfe0-95ab1391d419" containerName="payload-pre-processing" containerID="cri-o://cc401682207c2ee2ce2bcd708a47fb23d318681923efd79f0f962c1d210a69aa" gracePeriod=25
Oct 31 22:16:54.214432 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:54.214382    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:55.780526 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:55.780481    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-4bd51b-6b775cddf9-ld82r" podUID="d141422b-120e-4dc4-bfe0-95ab1391d419" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:16:55.780526 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:55.780509    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-4bd51b-7d76c5559f-r978k" podUID="8a302e86-54e3-4d9f-81ee-350498a9149e" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:16:56.005895 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:56.005857    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:57.732655 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:57.732616    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:59.176358 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:59.176320    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:59.794566 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:59.794520    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:16:59.925523 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:16:59.925487    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:00.034228 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:00.034193    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:00.466305 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:00.466267    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:01.434013 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:01.433978    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:01.997452 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:01.997390    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:02.106269 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:02.106228    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:02.215061 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:02.215025    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:02.594991 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:02.594926    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:03.314824 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:03.314777    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:04.058779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:04.058739    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:05.780107 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:05.780072    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-4bd51b-7d76c5559f-r978k" podUID="8a302e86-54e3-4d9f-81ee-350498a9149e" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:17:05.780513 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:05.780141    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-4bd51b-6b775cddf9-ld82r" podUID="d141422b-120e-4dc4-bfe0-95ab1391d419" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:17:05.848428 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:05.848394    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:06.314955 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:06.314908    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:06.427287 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:06.427251    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:06.880687 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:06.880652    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:06.997860 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:06.997820    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:07.187877 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:07.187769    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"]
Oct 31 22:17:07.188191 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:07.188045    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp" podUID="dd21505e-c4de-43eb-a5da-5c69c7c90785" containerName="istio-proxy" containerID="cri-o://4fa133c03e4b25c0f40660b51016cd018a651e4af7b98d465398898c656dd3de" gracePeriod=30
Oct 31 22:17:07.220210 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:07.220177    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:07.552620 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:07.552580    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:07.686111 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:07.686080    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:09.568664 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:09.568628    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:09.689986 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:09.689952    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:09.845031 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:09.844906    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"]
Oct 31 22:17:09.845202 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:09.845181    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8" podUID="20a6bd1d-39a8-4b3e-898b-bfdd13a155b3" containerName="istio-proxy" containerID="cri-o://860533e7e4c6596608c19aedf5a150b06b22f7804b651b043fe7f5eb8741b837" gracePeriod=30
Oct 31 22:17:09.880233 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:09.880200    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:11.581284 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:11.581247    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:12.477030 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:12.477005    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 22:17:12.630511 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:12.630472    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-istio-envoy\") pod \"dd21505e-c4de-43eb-a5da-5c69c7c90785\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") "
Oct 31 22:17:12.630951 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:12.630567    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-istio-data\") pod \"dd21505e-c4de-43eb-a5da-5c69c7c90785\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") "
Oct 31 22:17:12.630951 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:12.630612    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-workload-socket\") pod \"dd21505e-c4de-43eb-a5da-5c69c7c90785\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") "
Oct 31 22:17:12.630951 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:12.630640    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dd21505e-c4de-43eb-a5da-5c69c7c90785-istio-token\" (UniqueName: \"kubernetes.io/projected/dd21505e-c4de-43eb-a5da-5c69c7c90785-istio-token\") pod \"dd21505e-c4de-43eb-a5da-5c69c7c90785\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") "
Oct 31 22:17:12.630951 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:12.630667    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dd21505e-c4de-43eb-a5da-5c69c7c90785-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd21505e-c4de-43eb-a5da-5c69c7c90785-istiod-ca-cert\") pod \"dd21505e-c4de-43eb-a5da-5c69c7c90785\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") "
Oct 31 22:17:12.630951 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:12.630763    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-workload-certs\") pod \"dd21505e-c4de-43eb-a5da-5c69c7c90785\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") "
Oct 31 22:17:12.630951 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:12.630815    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-credential-socket\") pod \"dd21505e-c4de-43eb-a5da-5c69c7c90785\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") "
Oct 31 22:17:12.630951 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:12.630845    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-istio-data" pod "dd21505e-c4de-43eb-a5da-5c69c7c90785" (UID: "dd21505e-c4de-43eb-a5da-5c69c7c90785"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:17:12.630951 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:12.630901    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-workload-socket" pod "dd21505e-c4de-43eb-a5da-5c69c7c90785" (UID: "dd21505e-c4de-43eb-a5da-5c69c7c90785"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:17:12.631388 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:12.630968    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dd21505e-c4de-43eb-a5da-5c69c7c90785-kube-api-access-pf6jz\" (UniqueName: \"kubernetes.io/projected/dd21505e-c4de-43eb-a5da-5c69c7c90785-kube-api-access-pf6jz\") pod \"dd21505e-c4de-43eb-a5da-5c69c7c90785\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") "
Oct 31 22:17:12.631388 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:12.631072    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/dd21505e-c4de-43eb-a5da-5c69c7c90785-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd21505e-c4de-43eb-a5da-5c69c7c90785-istio-podinfo\") pod \"dd21505e-c4de-43eb-a5da-5c69c7c90785\" (UID: \"dd21505e-c4de-43eb-a5da-5c69c7c90785\") "
Oct 31 22:17:12.631388 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:12.631273    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd21505e-c4de-43eb-a5da-5c69c7c90785-istiod-ca-cert" pod "dd21505e-c4de-43eb-a5da-5c69c7c90785" (UID: "dd21505e-c4de-43eb-a5da-5c69c7c90785"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:17:12.631548 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:12.631459    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-workload-certs" pod "dd21505e-c4de-43eb-a5da-5c69c7c90785" (UID: "dd21505e-c4de-43eb-a5da-5c69c7c90785"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:17:12.631548 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:12.631500    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-credential-socket" pod "dd21505e-c4de-43eb-a5da-5c69c7c90785" (UID: "dd21505e-c4de-43eb-a5da-5c69c7c90785"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:17:12.631777 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:12.631756    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:12.631835 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:12.631786    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:12.631835 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:12.631802    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:12.631835 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:12.631818    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd21505e-c4de-43eb-a5da-5c69c7c90785-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:12.631934 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:12.631833    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:12.633020 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:12.632998    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd21505e-c4de-43eb-a5da-5c69c7c90785-istio-token" pod "dd21505e-c4de-43eb-a5da-5c69c7c90785" (UID: "dd21505e-c4de-43eb-a5da-5c69c7c90785"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:17:12.633259 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:12.633231    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd21505e-c4de-43eb-a5da-5c69c7c90785-kube-api-access-pf6jz" pod "dd21505e-c4de-43eb-a5da-5c69c7c90785" (UID: "dd21505e-c4de-43eb-a5da-5c69c7c90785"). InnerVolumeSpecName "kube-api-access-pf6jz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:17:12.633259 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:12.633234    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-istio-envoy" pod "dd21505e-c4de-43eb-a5da-5c69c7c90785" (UID: "dd21505e-c4de-43eb-a5da-5c69c7c90785"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:17:12.633430 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:12.633411    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dd21505e-c4de-43eb-a5da-5c69c7c90785-istio-podinfo" pod "dd21505e-c4de-43eb-a5da-5c69c7c90785" (UID: "dd21505e-c4de-43eb-a5da-5c69c7c90785"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 22:17:12.649892 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:12.649864    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"]
Oct 31 22:17:12.733487 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:12.733458    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd21505e-c4de-43eb-a5da-5c69c7c90785-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:12.733590 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:12.733489    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pf6jz\" (UniqueName: \"kubernetes.io/projected/dd21505e-c4de-43eb-a5da-5c69c7c90785-kube-api-access-pf6jz\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:12.733590 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:12.733499    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd21505e-c4de-43eb-a5da-5c69c7c90785-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:12.733590 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:12.733509    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd21505e-c4de-43eb-a5da-5c69c7c90785-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:12.778608 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:12.778572    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:12.944888 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:12.944858    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-4bd51b-7d76c5559f-r978k"
Oct 31 22:17:12.989713 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:12.989683    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-4bd51b-6b775cddf9-ld82r"
Oct 31 22:17:13.038837 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.038803    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8a302e86-54e3-4d9f-81ee-350498a9149e-kube-api-access-qbjgv\" (UniqueName: \"kubernetes.io/projected/8a302e86-54e3-4d9f-81ee-350498a9149e-kube-api-access-qbjgv\") pod \"8a302e86-54e3-4d9f-81ee-350498a9149e\" (UID: \"8a302e86-54e3-4d9f-81ee-350498a9149e\") "
Oct 31 22:17:13.039040 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.038890    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8a302e86-54e3-4d9f-81ee-350498a9149e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a302e86-54e3-4d9f-81ee-350498a9149e-plugins-config-volume\") pod \"8a302e86-54e3-4d9f-81ee-350498a9149e\" (UID: \"8a302e86-54e3-4d9f-81ee-350498a9149e\") "
Oct 31 22:17:13.039327 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.039304    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a302e86-54e3-4d9f-81ee-350498a9149e-plugins-config-volume" pod "8a302e86-54e3-4d9f-81ee-350498a9149e" (UID: "8a302e86-54e3-4d9f-81ee-350498a9149e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:17:13.039446 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.039429    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a302e86-54e3-4d9f-81ee-350498a9149e-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:13.041430 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.041407    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a302e86-54e3-4d9f-81ee-350498a9149e-kube-api-access-qbjgv" pod "8a302e86-54e3-4d9f-81ee-350498a9149e" (UID: "8a302e86-54e3-4d9f-81ee-350498a9149e"). InnerVolumeSpecName "kube-api-access-qbjgv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:17:13.071875 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.071836    4460 generic.go:356] "Generic (PLEG): container finished" podID="d141422b-120e-4dc4-bfe0-95ab1391d419" containerID="cc401682207c2ee2ce2bcd708a47fb23d318681923efd79f0f962c1d210a69aa" exitCode=0
Oct 31 22:17:13.072063 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.071895    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-4bd51b-6b775cddf9-ld82r"
Oct 31 22:17:13.072063 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.071929    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-4bd51b-6b775cddf9-ld82r" event={"ID":"d141422b-120e-4dc4-bfe0-95ab1391d419","Type":"ContainerDied","Data":"cc401682207c2ee2ce2bcd708a47fb23d318681923efd79f0f962c1d210a69aa"}
Oct 31 22:17:13.072195 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.072063    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-4bd51b-6b775cddf9-ld82r" event={"ID":"d141422b-120e-4dc4-bfe0-95ab1391d419","Type":"ContainerDied","Data":"811d7a5891fe7ee14951255435b5541b41f7799ad8aea342bfa396637d3c9f10"}
Oct 31 22:17:13.072195 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.072101    4460 scope.go:122] "RemoveContainer" containerID="cc401682207c2ee2ce2bcd708a47fb23d318681923efd79f0f962c1d210a69aa"
Oct 31 22:17:13.074279 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.074253    4460 generic.go:356] "Generic (PLEG): container finished" podID="dd21505e-c4de-43eb-a5da-5c69c7c90785" containerID="4fa133c03e4b25c0f40660b51016cd018a651e4af7b98d465398898c656dd3de" exitCode=0
Oct 31 22:17:13.074406 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.074349    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp" event={"ID":"dd21505e-c4de-43eb-a5da-5c69c7c90785","Type":"ContainerDied","Data":"4fa133c03e4b25c0f40660b51016cd018a651e4af7b98d465398898c656dd3de"}
Oct 31 22:17:13.074406 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.074401    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp" event={"ID":"dd21505e-c4de-43eb-a5da-5c69c7c90785","Type":"ContainerDied","Data":"dc33b3bed7cd3a0c88e83e8e9a27e509d56ab9974121f1f1f8ff7e61b0e581c2"}
Oct 31 22:17:13.074547 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.074474    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"
Oct 31 22:17:13.078889 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.078868    4460 generic.go:356] "Generic (PLEG): container finished" podID="8a302e86-54e3-4d9f-81ee-350498a9149e" containerID="d0d6a4d80081d692669527d2888b8d6d3d7a32a0b62eed694d1c8e1729c94cd6" exitCode=0
Oct 31 22:17:13.079017 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.078917    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-4bd51b-7d76c5559f-r978k"
Oct 31 22:17:13.079017 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.078982    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-4bd51b-7d76c5559f-r978k" event={"ID":"8a302e86-54e3-4d9f-81ee-350498a9149e","Type":"ContainerDied","Data":"d0d6a4d80081d692669527d2888b8d6d3d7a32a0b62eed694d1c8e1729c94cd6"}
Oct 31 22:17:13.079107 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.079039    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-4bd51b-7d76c5559f-r978k" event={"ID":"8a302e86-54e3-4d9f-81ee-350498a9149e","Type":"ContainerDied","Data":"5ad0f3a730f189e8e679885df3ed747bd9b2fced8e63453f6ec0755b851a41c2"}
Oct 31 22:17:13.088107 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.088081    4460 scope.go:122] "RemoveContainer" containerID="cc401682207c2ee2ce2bcd708a47fb23d318681923efd79f0f962c1d210a69aa"
Oct 31 22:17:13.088466 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:17:13.088439    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc401682207c2ee2ce2bcd708a47fb23d318681923efd79f0f962c1d210a69aa\": container with ID starting with cc401682207c2ee2ce2bcd708a47fb23d318681923efd79f0f962c1d210a69aa not found: ID does not exist" containerID="cc401682207c2ee2ce2bcd708a47fb23d318681923efd79f0f962c1d210a69aa"
Oct 31 22:17:13.088564 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.088474    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc401682207c2ee2ce2bcd708a47fb23d318681923efd79f0f962c1d210a69aa"} err="failed to get container status \"cc401682207c2ee2ce2bcd708a47fb23d318681923efd79f0f962c1d210a69aa\": rpc error: code = NotFound desc = could not find container \"cc401682207c2ee2ce2bcd708a47fb23d318681923efd79f0f962c1d210a69aa\": container with ID starting with cc401682207c2ee2ce2bcd708a47fb23d318681923efd79f0f962c1d210a69aa not found: ID does not exist"
Oct 31 22:17:13.088564 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.088494    4460 scope.go:122] "RemoveContainer" containerID="4fa133c03e4b25c0f40660b51016cd018a651e4af7b98d465398898c656dd3de"
Oct 31 22:17:13.099744 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.099718    4460 scope.go:122] "RemoveContainer" containerID="0012406ddf0805880548028d005622769699f85c53a707bed83322e99b8f90e2"
Oct 31 22:17:13.112529 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.112504    4460 scope.go:122] "RemoveContainer" containerID="4fa133c03e4b25c0f40660b51016cd018a651e4af7b98d465398898c656dd3de"
Oct 31 22:17:13.112866 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:17:13.112838    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fa133c03e4b25c0f40660b51016cd018a651e4af7b98d465398898c656dd3de\": container with ID starting with 4fa133c03e4b25c0f40660b51016cd018a651e4af7b98d465398898c656dd3de not found: ID does not exist" containerID="4fa133c03e4b25c0f40660b51016cd018a651e4af7b98d465398898c656dd3de"
Oct 31 22:17:13.112969 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.112880    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fa133c03e4b25c0f40660b51016cd018a651e4af7b98d465398898c656dd3de"} err="failed to get container status \"4fa133c03e4b25c0f40660b51016cd018a651e4af7b98d465398898c656dd3de\": rpc error: code = NotFound desc = could not find container \"4fa133c03e4b25c0f40660b51016cd018a651e4af7b98d465398898c656dd3de\": container with ID starting with 4fa133c03e4b25c0f40660b51016cd018a651e4af7b98d465398898c656dd3de not found: ID does not exist"
Oct 31 22:17:13.112969 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.112905    4460 scope.go:122] "RemoveContainer" containerID="0012406ddf0805880548028d005622769699f85c53a707bed83322e99b8f90e2"
Oct 31 22:17:13.113262 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:17:13.113238    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0012406ddf0805880548028d005622769699f85c53a707bed83322e99b8f90e2\": container with ID starting with 0012406ddf0805880548028d005622769699f85c53a707bed83322e99b8f90e2 not found: ID does not exist" containerID="0012406ddf0805880548028d005622769699f85c53a707bed83322e99b8f90e2"
Oct 31 22:17:13.113333 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.113274    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0012406ddf0805880548028d005622769699f85c53a707bed83322e99b8f90e2"} err="failed to get container status \"0012406ddf0805880548028d005622769699f85c53a707bed83322e99b8f90e2\": rpc error: code = NotFound desc = could not find container \"0012406ddf0805880548028d005622769699f85c53a707bed83322e99b8f90e2\": container with ID starting with 0012406ddf0805880548028d005622769699f85c53a707bed83322e99b8f90e2 not found: ID does not exist"
Oct 31 22:17:13.113333 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.113295    4460 scope.go:122] "RemoveContainer" containerID="d0d6a4d80081d692669527d2888b8d6d3d7a32a0b62eed694d1c8e1729c94cd6"
Oct 31 22:17:13.124478 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.124452    4460 scope.go:122] "RemoveContainer" containerID="d0d6a4d80081d692669527d2888b8d6d3d7a32a0b62eed694d1c8e1729c94cd6"
Oct 31 22:17:13.124756 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:17:13.124735    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0d6a4d80081d692669527d2888b8d6d3d7a32a0b62eed694d1c8e1729c94cd6\": container with ID starting with d0d6a4d80081d692669527d2888b8d6d3d7a32a0b62eed694d1c8e1729c94cd6 not found: ID does not exist" containerID="d0d6a4d80081d692669527d2888b8d6d3d7a32a0b62eed694d1c8e1729c94cd6"
Oct 31 22:17:13.124847 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.124760    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0d6a4d80081d692669527d2888b8d6d3d7a32a0b62eed694d1c8e1729c94cd6"} err="failed to get container status \"d0d6a4d80081d692669527d2888b8d6d3d7a32a0b62eed694d1c8e1729c94cd6\": rpc error: code = NotFound desc = could not find container \"d0d6a4d80081d692669527d2888b8d6d3d7a32a0b62eed694d1c8e1729c94cd6\": container with ID starting with d0d6a4d80081d692669527d2888b8d6d3d7a32a0b62eed694d1c8e1729c94cd6 not found: ID does not exist"
Oct 31 22:17:13.140531 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.140506    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d141422b-120e-4dc4-bfe0-95ab1391d419-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d141422b-120e-4dc4-bfe0-95ab1391d419-plugins-config-volume\") pod \"d141422b-120e-4dc4-bfe0-95ab1391d419\" (UID: \"d141422b-120e-4dc4-bfe0-95ab1391d419\") "
Oct 31 22:17:13.140644 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.140576    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d141422b-120e-4dc4-bfe0-95ab1391d419-kube-api-access-76v79\" (UniqueName: \"kubernetes.io/projected/d141422b-120e-4dc4-bfe0-95ab1391d419-kube-api-access-76v79\") pod \"d141422b-120e-4dc4-bfe0-95ab1391d419\" (UID: \"d141422b-120e-4dc4-bfe0-95ab1391d419\") "
Oct 31 22:17:13.140851 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.140831    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d141422b-120e-4dc4-bfe0-95ab1391d419-plugins-config-volume" pod "d141422b-120e-4dc4-bfe0-95ab1391d419" (UID: "d141422b-120e-4dc4-bfe0-95ab1391d419"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:17:13.141176 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.141159    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d141422b-120e-4dc4-bfe0-95ab1391d419-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:13.141219 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.141181    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbjgv\" (UniqueName: \"kubernetes.io/projected/8a302e86-54e3-4d9f-81ee-350498a9149e-kube-api-access-qbjgv\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:13.142828 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.142805    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d141422b-120e-4dc4-bfe0-95ab1391d419-kube-api-access-76v79" pod "d141422b-120e-4dc4-bfe0-95ab1391d419" (UID: "d141422b-120e-4dc4-bfe0-95ab1391d419"). InnerVolumeSpecName "kube-api-access-76v79". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:17:13.148444 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.148404    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"]
Oct 31 22:17:13.152516 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.152494    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-44405d-openshift-default-6447558b8c-2slxp"]
Oct 31 22:17:13.195969 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.195901    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-4bd51b-7d76c5559f-r978k"]
Oct 31 22:17:13.202548 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.202515    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-4bd51b-7d76c5559f-r978k"]
Oct 31 22:17:13.242275 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.242191    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76v79\" (UniqueName: \"kubernetes.io/projected/d141422b-120e-4dc4-bfe0-95ab1391d419-kube-api-access-76v79\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:13.358599 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.358565    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:13.436545 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.436503    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-4bd51b-6b775cddf9-ld82r"]
Oct 31 22:17:13.440785 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:13.440757    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-4bd51b-6b775cddf9-ld82r"]
Oct 31 22:17:14.159527 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:14.159488    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:14.227577 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:14.227542    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8a302e86-54e3-4d9f-81ee-350498a9149e" path="/var/lib/kubelet/pods/8a302e86-54e3-4d9f-81ee-350498a9149e/volumes"
Oct 31 22:17:14.228019 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:14.228004    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d141422b-120e-4dc4-bfe0-95ab1391d419" path="/var/lib/kubelet/pods/d141422b-120e-4dc4-bfe0-95ab1391d419/volumes"
Oct 31 22:17:14.228447 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:14.228434    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dd21505e-c4de-43eb-a5da-5c69c7c90785" path="/var/lib/kubelet/pods/dd21505e-c4de-43eb-a5da-5c69c7c90785/volumes"
Oct 31 22:17:14.275593 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:14.275523    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:15.103286 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:15.103248    4460 generic.go:356] "Generic (PLEG): container finished" podID="20a6bd1d-39a8-4b3e-898b-bfdd13a155b3" containerID="860533e7e4c6596608c19aedf5a150b06b22f7804b651b043fe7f5eb8741b837" exitCode=0
Oct 31 22:17:15.103486 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:15.103311    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8" event={"ID":"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3","Type":"ContainerDied","Data":"860533e7e4c6596608c19aedf5a150b06b22f7804b651b043fe7f5eb8741b837"}
Oct 31 22:17:15.103486 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:15.103366    4460 scope.go:122] "RemoveContainer" containerID="76cf8b03daa7ffbdbfec18e344a67a0b3659024fdafdcd4780bc651790b61d1d"
Oct 31 22:17:15.160569 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:15.160539    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 22:17:15.272142 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:15.272106    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-workload-certs\") pod \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") "
Oct 31 22:17:15.272368 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:15.272176    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istio-envoy\") pod \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") "
Oct 31 22:17:15.272368 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:15.272208    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istio-token\" (UniqueName: \"kubernetes.io/projected/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istio-token\") pod \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") "
Oct 31 22:17:15.272368 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:15.272306    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-kube-api-access-df2pd\" (UniqueName: \"kubernetes.io/projected/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-kube-api-access-df2pd\") pod \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") "
Oct 31 22:17:15.272368 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:15.272331    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-workload-socket\") pod \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") "
Oct 31 22:17:15.272596 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:15.272382    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-credential-socket\") pod \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") "
Oct 31 22:17:15.272596 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:15.272462    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istiod-ca-cert\") pod \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") "
Oct 31 22:17:15.272596 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:15.272477    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-workload-certs" pod "20a6bd1d-39a8-4b3e-898b-bfdd13a155b3" (UID: "20a6bd1d-39a8-4b3e-898b-bfdd13a155b3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:17:15.272596 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:15.272555    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istio-podinfo\") pod \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") "
Oct 31 22:17:15.272801 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:15.272649    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istio-data\") pod \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\" (UID: \"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3\") "
Oct 31 22:17:15.272801 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:15.272693    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-workload-socket" pod "20a6bd1d-39a8-4b3e-898b-bfdd13a155b3" (UID: "20a6bd1d-39a8-4b3e-898b-bfdd13a155b3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:17:15.272903 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:15.272870    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istiod-ca-cert" pod "20a6bd1d-39a8-4b3e-898b-bfdd13a155b3" (UID: "20a6bd1d-39a8-4b3e-898b-bfdd13a155b3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:17:15.272903 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:15.272895    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-credential-socket" pod "20a6bd1d-39a8-4b3e-898b-bfdd13a155b3" (UID: "20a6bd1d-39a8-4b3e-898b-bfdd13a155b3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:17:15.273088 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:15.273063    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istio-data" pod "20a6bd1d-39a8-4b3e-898b-bfdd13a155b3" (UID: "20a6bd1d-39a8-4b3e-898b-bfdd13a155b3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:17:15.273426 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:15.273402    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:15.273532 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:15.273431    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:15.273532 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:15.273447    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:15.273532 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:15.273460    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:15.273532 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:15.273475    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:15.274778 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:15.274746    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istio-envoy" pod "20a6bd1d-39a8-4b3e-898b-bfdd13a155b3" (UID: "20a6bd1d-39a8-4b3e-898b-bfdd13a155b3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:17:15.274860 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:15.274826    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istio-token" pod "20a6bd1d-39a8-4b3e-898b-bfdd13a155b3" (UID: "20a6bd1d-39a8-4b3e-898b-bfdd13a155b3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:17:15.275008 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:15.274990    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-kube-api-access-df2pd" pod "20a6bd1d-39a8-4b3e-898b-bfdd13a155b3" (UID: "20a6bd1d-39a8-4b3e-898b-bfdd13a155b3"). InnerVolumeSpecName "kube-api-access-df2pd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:17:15.275261 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:15.275243    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istio-podinfo" pod "20a6bd1d-39a8-4b3e-898b-bfdd13a155b3" (UID: "20a6bd1d-39a8-4b3e-898b-bfdd13a155b3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 22:17:15.319413 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:15.319378    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:15.381694 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:15.381069    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:15.381694 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:15.381109    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:15.381694 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:15.381123    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-df2pd\" (UniqueName: \"kubernetes.io/projected/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-kube-api-access-df2pd\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:15.381694 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:15.381141    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:16.111138 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:16.111105    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"
Oct 31 22:17:16.111348 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:16.111141    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8" event={"ID":"20a6bd1d-39a8-4b3e-898b-bfdd13a155b3","Type":"ContainerDied","Data":"f1c5127b2f628179f01f792868624f1372f02e9db0f0f78b4af62092d969e618"}
Oct 31 22:17:16.111348 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:16.111202    4460 scope.go:122] "RemoveContainer" containerID="860533e7e4c6596608c19aedf5a150b06b22f7804b651b043fe7f5eb8741b837"
Oct 31 22:17:16.183113 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:16.183078    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"]
Oct 31 22:17:16.196613 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:16.196584    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-4bd51b-openshift-default-b8544dc4f-jrkg8"]
Oct 31 22:17:16.226054 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:16.226026    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="20a6bd1d-39a8-4b3e-898b-bfdd13a155b3" path="/var/lib/kubelet/pods/20a6bd1d-39a8-4b3e-898b-bfdd13a155b3/volumes"
Oct 31 22:17:17.034167 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:17.034136    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:18.380423 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:18.380381    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-c16684-685b865b6c-qbtqr"]
Oct 31 22:17:18.413548 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:18.413506    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-c16684-94dc7cbd7-gjj44"]
Oct 31 22:17:18.522273 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:18.522221    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:18.638029 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:18.637799    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:18.829870 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:18.829827    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:20.556197 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:20.556155    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:20.878698 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:20.878601    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-c34cbd-559d4ccccf-bxss4"]
Oct 31 22:17:20.912103 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:20.912068    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-c34cbd-7dc696c947-v5wpt"]
Oct 31 22:17:21.024721 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:21.024674    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:21.170679 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:21.170581    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:21.453265 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:17:21.453174    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:17:22.428393 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:22.428359    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:23.381497 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:23.381452    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w2-c16684-685b865b6c-qbtqr" podUID="5ce9718e-7d7b-4fc4-b506-b25cd880efc3" containerName="payload-processing" containerID="cri-o://a64a5862408480fe92d984ec50e36cbae130e0ad181b98ac1ba7b388f3210fa1" gracePeriod=25
Oct 31 22:17:23.414780 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:23.414724    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c16684-94dc7cbd7-gjj44" podUID="076218ca-4ba5-43dc-967e-fb30eda6e029" containerName="payload-pre-processing" containerID="cri-o://a73559c973889aff2df223ab52e35a5f92a5f300f4069a71761fa1ff23333c6d" gracePeriod=25
Oct 31 22:17:23.960994 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:23.960951    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:24.505758 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:24.505721    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:24.614480 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:24.614445    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:24.723339 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:24.723305    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:25.121464 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:25.121430    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:25.879431 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:25.879383    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-b-c34cbd-559d4ccccf-bxss4" podUID="e2e149ab-f40b-4d96-91df-b230dfae9e90" containerName="payload-processing" containerID="cri-o://8f7e815b1a572b6f9ed988032d688cd0868ea44374f15808e8082a7d3d093825" gracePeriod=25
Oct 31 22:17:25.913377 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:25.913338    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c34cbd-7dc696c947-v5wpt" podUID="a22daeb7-8d30-4dcc-98be-e8f404ab1c42" containerName="payload-pre-processing" containerID="cri-o://821788ddfbf583e3a0219e06a757dda353f9ccd1dd26540ccc71c7a9f3be5a5b" gracePeriod=25
Oct 31 22:17:26.466880 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:26.466837    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:26.997620 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:26.997575    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 22:17:27.326772 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:27.326737    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-b-c34cbd-559d4ccccf-bxss4" podUID="e2e149ab-f40b-4d96-91df-b230dfae9e90" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:17:27.326970 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:27.326737    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c34cbd-7dc696c947-v5wpt" podUID="a22daeb7-8d30-4dcc-98be-e8f404ab1c42" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:17:27.672889 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:27.672796    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz" podUID="93a42128-ee74-4dcd-b158-27e33cfd44f9" containerName="main" containerID="cri-o://1ed2052e514627cb81374a78fd18e02589b6dffb1424d02405ab49435aa25ff6" gracePeriod=45
Oct 31 22:17:27.677646 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:27.677611    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz" podUID="93a42128-ee74-4dcd-b158-27e33cfd44f9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.67:8000/ready\": dial tcp 10.133.0.67:8000: connect: connection refused"
Oct 31 22:17:27.990212 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:27.990186    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 22:17:28.078405 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:28.078362    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93a42128-ee74-4dcd-b158-27e33cfd44f9-dshm\" (UniqueName: \"kubernetes.io/empty-dir/93a42128-ee74-4dcd-b158-27e33cfd44f9-dshm\") pod \"93a42128-ee74-4dcd-b158-27e33cfd44f9\" (UID: \"93a42128-ee74-4dcd-b158-27e33cfd44f9\") "
Oct 31 22:17:28.078618 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:28.078423    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93a42128-ee74-4dcd-b158-27e33cfd44f9-home\" (UniqueName: \"kubernetes.io/empty-dir/93a42128-ee74-4dcd-b158-27e33cfd44f9-home\") pod \"93a42128-ee74-4dcd-b158-27e33cfd44f9\" (UID: \"93a42128-ee74-4dcd-b158-27e33cfd44f9\") "
Oct 31 22:17:28.078618 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:28.078576    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93a42128-ee74-4dcd-b158-27e33cfd44f9-kube-api-access-cjh6f\" (UniqueName: \"kubernetes.io/projected/93a42128-ee74-4dcd-b158-27e33cfd44f9-kube-api-access-cjh6f\") pod \"93a42128-ee74-4dcd-b158-27e33cfd44f9\" (UID: \"93a42128-ee74-4dcd-b158-27e33cfd44f9\") "
Oct 31 22:17:28.078765 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:28.078680    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93a42128-ee74-4dcd-b158-27e33cfd44f9-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93a42128-ee74-4dcd-b158-27e33cfd44f9-model-cache\") pod \"93a42128-ee74-4dcd-b158-27e33cfd44f9\" (UID: \"93a42128-ee74-4dcd-b158-27e33cfd44f9\") "
Oct 31 22:17:28.078765 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:28.078723    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93a42128-ee74-4dcd-b158-27e33cfd44f9-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93a42128-ee74-4dcd-b158-27e33cfd44f9-tmp-dir\") pod \"93a42128-ee74-4dcd-b158-27e33cfd44f9\" (UID: \"93a42128-ee74-4dcd-b158-27e33cfd44f9\") "
Oct 31 22:17:28.078886 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:28.078809    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/93a42128-ee74-4dcd-b158-27e33cfd44f9-tls-certs\" (UniqueName: \"kubernetes.io/secret/93a42128-ee74-4dcd-b158-27e33cfd44f9-tls-certs\") pod \"93a42128-ee74-4dcd-b158-27e33cfd44f9\" (UID: \"93a42128-ee74-4dcd-b158-27e33cfd44f9\") "
Oct 31 22:17:28.079132 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:28.079061    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93a42128-ee74-4dcd-b158-27e33cfd44f9-tmp-dir" pod "93a42128-ee74-4dcd-b158-27e33cfd44f9" (UID: "93a42128-ee74-4dcd-b158-27e33cfd44f9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:17:28.079643 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:28.079491    4460 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93a42128-ee74-4dcd-b158-27e33cfd44f9-tmp-dir\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:28.079643 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:28.079594    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93a42128-ee74-4dcd-b158-27e33cfd44f9-model-cache" pod "93a42128-ee74-4dcd-b158-27e33cfd44f9" (UID: "93a42128-ee74-4dcd-b158-27e33cfd44f9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:17:28.080130 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:28.080082    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93a42128-ee74-4dcd-b158-27e33cfd44f9-home" pod "93a42128-ee74-4dcd-b158-27e33cfd44f9" (UID: "93a42128-ee74-4dcd-b158-27e33cfd44f9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:17:28.087084 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:28.087055    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93a42128-ee74-4dcd-b158-27e33cfd44f9-dshm" pod "93a42128-ee74-4dcd-b158-27e33cfd44f9" (UID: "93a42128-ee74-4dcd-b158-27e33cfd44f9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:17:28.087205 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:28.087113    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93a42128-ee74-4dcd-b158-27e33cfd44f9-kube-api-access-cjh6f" pod "93a42128-ee74-4dcd-b158-27e33cfd44f9" (UID: "93a42128-ee74-4dcd-b158-27e33cfd44f9"). InnerVolumeSpecName "kube-api-access-cjh6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:17:28.095441 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:28.095360    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93a42128-ee74-4dcd-b158-27e33cfd44f9-tls-certs" pod "93a42128-ee74-4dcd-b158-27e33cfd44f9" (UID: "93a42128-ee74-4dcd-b158-27e33cfd44f9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 22:17:28.180655 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:28.180621    4460 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93a42128-ee74-4dcd-b158-27e33cfd44f9-tls-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:28.180655 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:28.180655    4460 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93a42128-ee74-4dcd-b158-27e33cfd44f9-dshm\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:28.180866 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:28.180666    4460 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93a42128-ee74-4dcd-b158-27e33cfd44f9-home\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:28.180866 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:28.180675    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjh6f\" (UniqueName: \"kubernetes.io/projected/93a42128-ee74-4dcd-b158-27e33cfd44f9-kube-api-access-cjh6f\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:28.180866 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:28.180695    4460 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93a42128-ee74-4dcd-b158-27e33cfd44f9-model-cache\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:28.230541 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:28.230447    4460 generic.go:356] "Generic (PLEG): container finished" podID="93a42128-ee74-4dcd-b158-27e33cfd44f9" containerID="1ed2052e514627cb81374a78fd18e02589b6dffb1424d02405ab49435aa25ff6" exitCode=0
Oct 31 22:17:28.230541 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:28.230497    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"
Oct 31 22:17:28.230541 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:28.230518    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz" event={"ID":"93a42128-ee74-4dcd-b158-27e33cfd44f9","Type":"ContainerDied","Data":"1ed2052e514627cb81374a78fd18e02589b6dffb1424d02405ab49435aa25ff6"}
Oct 31 22:17:28.230822 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:28.230546    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz" event={"ID":"93a42128-ee74-4dcd-b158-27e33cfd44f9","Type":"ContainerDied","Data":"628f96f122bd76ec6ced13b47110682c74932e81e7b8ab70388403f798dd921b"}
Oct 31 22:17:28.230822 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:28.230569    4460 scope.go:122] "RemoveContainer" containerID="1ed2052e514627cb81374a78fd18e02589b6dffb1424d02405ab49435aa25ff6"
Oct 31 22:17:28.241643 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:28.241622    4460 scope.go:122] "RemoveContainer" containerID="1ed2052e514627cb81374a78fd18e02589b6dffb1424d02405ab49435aa25ff6"
Oct 31 22:17:28.241889 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:17:28.241866    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ed2052e514627cb81374a78fd18e02589b6dffb1424d02405ab49435aa25ff6\": container with ID starting with 1ed2052e514627cb81374a78fd18e02589b6dffb1424d02405ab49435aa25ff6 not found: ID does not exist" containerID="1ed2052e514627cb81374a78fd18e02589b6dffb1424d02405ab49435aa25ff6"
Oct 31 22:17:28.241934 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:28.241897    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ed2052e514627cb81374a78fd18e02589b6dffb1424d02405ab49435aa25ff6"} err="failed to get container status \"1ed2052e514627cb81374a78fd18e02589b6dffb1424d02405ab49435aa25ff6\": rpc error: code = NotFound desc = could not find container \"1ed2052e514627cb81374a78fd18e02589b6dffb1424d02405ab49435aa25ff6\": container with ID starting with 1ed2052e514627cb81374a78fd18e02589b6dffb1424d02405ab49435aa25ff6 not found: ID does not exist"
Oct 31 22:17:28.248721 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:28.248699    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:28.292284 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:28.292213    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"]
Oct 31 22:17:28.324220 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:28.324181    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-c16684/distinct-2-w2-c16684-kserve-97bdb9bb8-6jzvz"]
Oct 31 22:17:28.394076 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:28.394037    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:29.938717 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:29.938678    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:30.228342 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:30.227022    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="93a42128-ee74-4dcd-b158-27e33cfd44f9" path="/var/lib/kubelet/pods/93a42128-ee74-4dcd-b158-27e33cfd44f9/volumes"
Oct 31 22:17:31.307063 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:31.307022    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-c16684-685b865b6c-qbtqr" podUID="5ce9718e-7d7b-4fc4-b506-b25cd880efc3" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:17:31.475095 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:31.475062    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:31.583862 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:31.583769    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:31.702908 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:31.696460    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:31.766039 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:31.766002    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"]
Oct 31 22:17:31.766317 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:31.766287    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg" podUID="be5c2058-2c58-4394-a4e9-c02e2aa18073" containerName="istio-proxy" containerID="cri-o://f6728ff416cceecec306dcafa426796dd99655792753ad82ab80d5fe451e6794" gracePeriod=30
Oct 31 22:17:31.810527 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:31.810480    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:32.313687 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:32.313652    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c16684-94dc7cbd7-gjj44" podUID="076218ca-4ba5-43dc-967e-fb30eda6e029" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:17:33.520373 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:33.520339    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:35.274584 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:35.274545    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:36.744214 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:36.744174    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:37.187767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.187733    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:17:37.256226 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.256168    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:37.293833 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.293795    4460 generic.go:356] "Generic (PLEG): container finished" podID="be5c2058-2c58-4394-a4e9-c02e2aa18073" containerID="f6728ff416cceecec306dcafa426796dd99655792753ad82ab80d5fe451e6794" exitCode=0
Oct 31 22:17:37.294020 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.293856    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg" event={"ID":"be5c2058-2c58-4394-a4e9-c02e2aa18073","Type":"ContainerDied","Data":"f6728ff416cceecec306dcafa426796dd99655792753ad82ab80d5fe451e6794"}
Oct 31 22:17:37.294020 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.293908    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg" event={"ID":"be5c2058-2c58-4394-a4e9-c02e2aa18073","Type":"ContainerDied","Data":"6c2917f5738b2825165aef200f833155eca0910e3868038e53a83149746a6f4f"}
Oct 31 22:17:37.294020 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.293972    4460 scope.go:122] "RemoveContainer" containerID="f6728ff416cceecec306dcafa426796dd99655792753ad82ab80d5fe451e6794"
Oct 31 22:17:37.294154 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.294137    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"
Oct 31 22:17:37.309866 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.309842    4460 scope.go:122] "RemoveContainer" containerID="d3d6f36ab726c7a88427c4ae544f4bf8603dcbc37e179a1674613a3f31549f91"
Oct 31 22:17:37.320977 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.320950    4460 scope.go:122] "RemoveContainer" containerID="f6728ff416cceecec306dcafa426796dd99655792753ad82ab80d5fe451e6794"
Oct 31 22:17:37.321278 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:17:37.321256    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6728ff416cceecec306dcafa426796dd99655792753ad82ab80d5fe451e6794\": container with ID starting with f6728ff416cceecec306dcafa426796dd99655792753ad82ab80d5fe451e6794 not found: ID does not exist" containerID="f6728ff416cceecec306dcafa426796dd99655792753ad82ab80d5fe451e6794"
Oct 31 22:17:37.321341 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.321287    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6728ff416cceecec306dcafa426796dd99655792753ad82ab80d5fe451e6794"} err="failed to get container status \"f6728ff416cceecec306dcafa426796dd99655792753ad82ab80d5fe451e6794\": rpc error: code = NotFound desc = could not find container \"f6728ff416cceecec306dcafa426796dd99655792753ad82ab80d5fe451e6794\": container with ID starting with f6728ff416cceecec306dcafa426796dd99655792753ad82ab80d5fe451e6794 not found: ID does not exist"
Oct 31 22:17:37.321341 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.321303    4460 scope.go:122] "RemoveContainer" containerID="d3d6f36ab726c7a88427c4ae544f4bf8603dcbc37e179a1674613a3f31549f91"
Oct 31 22:17:37.321570 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:17:37.321549    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3d6f36ab726c7a88427c4ae544f4bf8603dcbc37e179a1674613a3f31549f91\": container with ID starting with d3d6f36ab726c7a88427c4ae544f4bf8603dcbc37e179a1674613a3f31549f91 not found: ID does not exist" containerID="d3d6f36ab726c7a88427c4ae544f4bf8603dcbc37e179a1674613a3f31549f91"
Oct 31 22:17:37.321629 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.321580    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3d6f36ab726c7a88427c4ae544f4bf8603dcbc37e179a1674613a3f31549f91"} err="failed to get container status \"d3d6f36ab726c7a88427c4ae544f4bf8603dcbc37e179a1674613a3f31549f91\": rpc error: code = NotFound desc = could not find container \"d3d6f36ab726c7a88427c4ae544f4bf8603dcbc37e179a1674613a3f31549f91\": container with ID starting with d3d6f36ab726c7a88427c4ae544f4bf8603dcbc37e179a1674613a3f31549f91 not found: ID does not exist"
Oct 31 22:17:37.322950 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.322921    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-credential-socket\") pod \"be5c2058-2c58-4394-a4e9-c02e2aa18073\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") "
Oct 31 22:17:37.323025 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.322996    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/be5c2058-2c58-4394-a4e9-c02e2aa18073-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be5c2058-2c58-4394-a4e9-c02e2aa18073-istiod-ca-cert\") pod \"be5c2058-2c58-4394-a4e9-c02e2aa18073\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") "
Oct 31 22:17:37.323152 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.323131    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-credential-socket" pod "be5c2058-2c58-4394-a4e9-c02e2aa18073" (UID: "be5c2058-2c58-4394-a4e9-c02e2aa18073"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:17:37.323232 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.323148    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-workload-certs\") pod \"be5c2058-2c58-4394-a4e9-c02e2aa18073\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") "
Oct 31 22:17:37.323232 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.323198    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/be5c2058-2c58-4394-a4e9-c02e2aa18073-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be5c2058-2c58-4394-a4e9-c02e2aa18073-istio-podinfo\") pod \"be5c2058-2c58-4394-a4e9-c02e2aa18073\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") "
Oct 31 22:17:37.323362 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.323319    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/be5c2058-2c58-4394-a4e9-c02e2aa18073-kube-api-access-qlmxp\" (UniqueName: \"kubernetes.io/projected/be5c2058-2c58-4394-a4e9-c02e2aa18073-kube-api-access-qlmxp\") pod \"be5c2058-2c58-4394-a4e9-c02e2aa18073\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") "
Oct 31 22:17:37.323362 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.323348    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/be5c2058-2c58-4394-a4e9-c02e2aa18073-istio-token\" (UniqueName: \"kubernetes.io/projected/be5c2058-2c58-4394-a4e9-c02e2aa18073-istio-token\") pod \"be5c2058-2c58-4394-a4e9-c02e2aa18073\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") "
Oct 31 22:17:37.323469 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.323357    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-workload-certs" pod "be5c2058-2c58-4394-a4e9-c02e2aa18073" (UID: "be5c2058-2c58-4394-a4e9-c02e2aa18073"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:17:37.323469 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.323374    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-istio-envoy\") pod \"be5c2058-2c58-4394-a4e9-c02e2aa18073\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") "
Oct 31 22:17:37.323469 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.323405    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be5c2058-2c58-4394-a4e9-c02e2aa18073-istiod-ca-cert" pod "be5c2058-2c58-4394-a4e9-c02e2aa18073" (UID: "be5c2058-2c58-4394-a4e9-c02e2aa18073"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:17:37.323469 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.323430    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-workload-socket\") pod \"be5c2058-2c58-4394-a4e9-c02e2aa18073\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") "
Oct 31 22:17:37.323696 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.323513    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-istio-data\") pod \"be5c2058-2c58-4394-a4e9-c02e2aa18073\" (UID: \"be5c2058-2c58-4394-a4e9-c02e2aa18073\") "
Oct 31 22:17:37.324625 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.323762    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-workload-socket" pod "be5c2058-2c58-4394-a4e9-c02e2aa18073" (UID: "be5c2058-2c58-4394-a4e9-c02e2aa18073"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:17:37.324625 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.324320    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-istio-data" pod "be5c2058-2c58-4394-a4e9-c02e2aa18073" (UID: "be5c2058-2c58-4394-a4e9-c02e2aa18073"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:17:37.324625 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.324519    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:37.324625 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.324538    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be5c2058-2c58-4394-a4e9-c02e2aa18073-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:37.324625 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.324553    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:37.324625 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.324567    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:37.325876 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.325852    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/be5c2058-2c58-4394-a4e9-c02e2aa18073-istio-podinfo" pod "be5c2058-2c58-4394-a4e9-c02e2aa18073" (UID: "be5c2058-2c58-4394-a4e9-c02e2aa18073"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 22:17:37.326036 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.326019    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be5c2058-2c58-4394-a4e9-c02e2aa18073-kube-api-access-qlmxp" pod "be5c2058-2c58-4394-a4e9-c02e2aa18073" (UID: "be5c2058-2c58-4394-a4e9-c02e2aa18073"). InnerVolumeSpecName "kube-api-access-qlmxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:17:37.326235 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.326211    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be5c2058-2c58-4394-a4e9-c02e2aa18073-istio-token" pod "be5c2058-2c58-4394-a4e9-c02e2aa18073" (UID: "be5c2058-2c58-4394-a4e9-c02e2aa18073"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:17:37.326327 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.326240    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-istio-envoy" pod "be5c2058-2c58-4394-a4e9-c02e2aa18073" (UID: "be5c2058-2c58-4394-a4e9-c02e2aa18073"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:17:37.327270 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.327243    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-b-c34cbd-559d4ccccf-bxss4" podUID="e2e149ab-f40b-4d96-91df-b230dfae9e90" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:17:37.327377 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.327238    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c34cbd-7dc696c947-v5wpt" podUID="a22daeb7-8d30-4dcc-98be-e8f404ab1c42" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:17:37.374463 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.374442    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:37.425970 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.425860    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlmxp\" (UniqueName: \"kubernetes.io/projected/be5c2058-2c58-4394-a4e9-c02e2aa18073-kube-api-access-qlmxp\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:37.425970 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.425903    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be5c2058-2c58-4394-a4e9-c02e2aa18073-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:37.425970 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.425914    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:37.425970 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.425921    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be5c2058-2c58-4394-a4e9-c02e2aa18073-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:37.425970 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.425929    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be5c2058-2c58-4394-a4e9-c02e2aa18073-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:37.645613 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.645579    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"]
Oct 31 22:17:37.659087 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.659057    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-cd421c-openshift-default-5b7f98b7b5-mnhzg"]
Oct 31 22:17:37.855675 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:37.855637    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:38.225149 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:38.225074    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="be5c2058-2c58-4394-a4e9-c02e2aa18073" path="/var/lib/kubelet/pods/be5c2058-2c58-4394-a4e9-c02e2aa18073/volumes"
Oct 31 22:17:39.230090 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:39.230052    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:41.040709 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:41.040667    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:41.307370 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:41.307274    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-c16684-685b865b6c-qbtqr" podUID="5ce9718e-7d7b-4fc4-b506-b25cd880efc3" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:17:42.312980 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:42.312927    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c16684-94dc7cbd7-gjj44" podUID="076218ca-4ba5-43dc-967e-fb30eda6e029" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:17:42.844441 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:42.844404    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c08360-58948b75c9-k66nk"]
Oct 31 22:17:42.845069 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:42.845040    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:42.877723 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:42.877695    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-c08360-8645d7ff59-s2tn2"]
Oct 31 22:17:42.985582 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:42.985547    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:43.110439 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:43.110348    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:43.739342 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:43.739319    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c16684-94dc7cbd7-gjj44"
Oct 31 22:17:43.769639 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:43.769617    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-c16684-685b865b6c-qbtqr"
Oct 31 22:17:44.021527 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:44.021489    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/076218ca-4ba5-43dc-967e-fb30eda6e029-kube-api-access-lmdgz\" (UniqueName: \"kubernetes.io/projected/076218ca-4ba5-43dc-967e-fb30eda6e029-kube-api-access-lmdgz\") pod \"076218ca-4ba5-43dc-967e-fb30eda6e029\" (UID: \"076218ca-4ba5-43dc-967e-fb30eda6e029\") "
Oct 31 22:17:44.021735 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:44.021567    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/076218ca-4ba5-43dc-967e-fb30eda6e029-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/076218ca-4ba5-43dc-967e-fb30eda6e029-plugins-config-volume\") pod \"076218ca-4ba5-43dc-967e-fb30eda6e029\" (UID: \"076218ca-4ba5-43dc-967e-fb30eda6e029\") "
Oct 31 22:17:44.021735 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:44.021631    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5ce9718e-7d7b-4fc4-b506-b25cd880efc3-kube-api-access-2chtv\" (UniqueName: \"kubernetes.io/projected/5ce9718e-7d7b-4fc4-b506-b25cd880efc3-kube-api-access-2chtv\") pod \"5ce9718e-7d7b-4fc4-b506-b25cd880efc3\" (UID: \"5ce9718e-7d7b-4fc4-b506-b25cd880efc3\") "
Oct 31 22:17:44.021735 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:44.021713    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5ce9718e-7d7b-4fc4-b506-b25cd880efc3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ce9718e-7d7b-4fc4-b506-b25cd880efc3-plugins-config-volume\") pod \"5ce9718e-7d7b-4fc4-b506-b25cd880efc3\" (UID: \"5ce9718e-7d7b-4fc4-b506-b25cd880efc3\") "
Oct 31 22:17:44.022113 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:44.022082    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ce9718e-7d7b-4fc4-b506-b25cd880efc3-plugins-config-volume" pod "5ce9718e-7d7b-4fc4-b506-b25cd880efc3" (UID: "5ce9718e-7d7b-4fc4-b506-b25cd880efc3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:17:44.022113 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:44.022080    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/076218ca-4ba5-43dc-967e-fb30eda6e029-plugins-config-volume" pod "076218ca-4ba5-43dc-967e-fb30eda6e029" (UID: "076218ca-4ba5-43dc-967e-fb30eda6e029"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:17:44.022368 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:44.022293    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/076218ca-4ba5-43dc-967e-fb30eda6e029-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:44.022368 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:44.022316    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ce9718e-7d7b-4fc4-b506-b25cd880efc3-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:44.023859 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:44.023835    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ce9718e-7d7b-4fc4-b506-b25cd880efc3-kube-api-access-2chtv" pod "5ce9718e-7d7b-4fc4-b506-b25cd880efc3" (UID: "5ce9718e-7d7b-4fc4-b506-b25cd880efc3"). InnerVolumeSpecName "kube-api-access-2chtv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:17:44.024042 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:44.023886    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/076218ca-4ba5-43dc-967e-fb30eda6e029-kube-api-access-lmdgz" pod "076218ca-4ba5-43dc-967e-fb30eda6e029" (UID: "076218ca-4ba5-43dc-967e-fb30eda6e029"). InnerVolumeSpecName "kube-api-access-lmdgz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:17:44.136973 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:44.131351    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2chtv\" (UniqueName: \"kubernetes.io/projected/5ce9718e-7d7b-4fc4-b506-b25cd880efc3-kube-api-access-2chtv\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:44.136973 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:44.131406    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmdgz\" (UniqueName: \"kubernetes.io/projected/076218ca-4ba5-43dc-967e-fb30eda6e029-kube-api-access-lmdgz\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:44.342321 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:44.342230    4460 generic.go:356] "Generic (PLEG): container finished" podID="5ce9718e-7d7b-4fc4-b506-b25cd880efc3" containerID="a64a5862408480fe92d984ec50e36cbae130e0ad181b98ac1ba7b388f3210fa1" exitCode=0
Oct 31 22:17:44.342321 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:44.342272    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-c16684-685b865b6c-qbtqr"
Oct 31 22:17:44.342546 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:44.342311    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-c16684-685b865b6c-qbtqr" event={"ID":"5ce9718e-7d7b-4fc4-b506-b25cd880efc3","Type":"ContainerDied","Data":"a64a5862408480fe92d984ec50e36cbae130e0ad181b98ac1ba7b388f3210fa1"}
Oct 31 22:17:44.342546 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:44.342368    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-c16684-685b865b6c-qbtqr" event={"ID":"5ce9718e-7d7b-4fc4-b506-b25cd880efc3","Type":"ContainerDied","Data":"89645908b2bc753769f43ad47aec0845ddd75c6ea50e9aa93278aab9c034972c"}
Oct 31 22:17:44.342546 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:44.342395    4460 scope.go:122] "RemoveContainer" containerID="a64a5862408480fe92d984ec50e36cbae130e0ad181b98ac1ba7b388f3210fa1"
Oct 31 22:17:44.344063 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:44.344039    4460 generic.go:356] "Generic (PLEG): container finished" podID="076218ca-4ba5-43dc-967e-fb30eda6e029" containerID="a73559c973889aff2df223ab52e35a5f92a5f300f4069a71761fa1ff23333c6d" exitCode=0
Oct 31 22:17:44.344195 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:44.344101    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c16684-94dc7cbd7-gjj44"
Oct 31 22:17:44.344395 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:44.344106    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c16684-94dc7cbd7-gjj44" event={"ID":"076218ca-4ba5-43dc-967e-fb30eda6e029","Type":"ContainerDied","Data":"a73559c973889aff2df223ab52e35a5f92a5f300f4069a71761fa1ff23333c6d"}
Oct 31 22:17:44.344395 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:44.344363    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c16684-94dc7cbd7-gjj44" event={"ID":"076218ca-4ba5-43dc-967e-fb30eda6e029","Type":"ContainerDied","Data":"897f3d98fad18551cb35ec5d998a6d54391aa254695f9536b8ab01362b118e96"}
Oct 31 22:17:44.352799 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:44.352779    4460 scope.go:122] "RemoveContainer" containerID="a64a5862408480fe92d984ec50e36cbae130e0ad181b98ac1ba7b388f3210fa1"
Oct 31 22:17:44.353122 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:17:44.353099    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a64a5862408480fe92d984ec50e36cbae130e0ad181b98ac1ba7b388f3210fa1\": container with ID starting with a64a5862408480fe92d984ec50e36cbae130e0ad181b98ac1ba7b388f3210fa1 not found: ID does not exist" containerID="a64a5862408480fe92d984ec50e36cbae130e0ad181b98ac1ba7b388f3210fa1"
Oct 31 22:17:44.353205 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:44.353129    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a64a5862408480fe92d984ec50e36cbae130e0ad181b98ac1ba7b388f3210fa1"} err="failed to get container status \"a64a5862408480fe92d984ec50e36cbae130e0ad181b98ac1ba7b388f3210fa1\": rpc error: code = NotFound desc = could not find container \"a64a5862408480fe92d984ec50e36cbae130e0ad181b98ac1ba7b388f3210fa1\": container with ID starting with a64a5862408480fe92d984ec50e36cbae130e0ad181b98ac1ba7b388f3210fa1 not found: ID does not exist"
Oct 31 22:17:44.353205 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:44.353146    4460 scope.go:122] "RemoveContainer" containerID="a73559c973889aff2df223ab52e35a5f92a5f300f4069a71761fa1ff23333c6d"
Oct 31 22:17:44.364105 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:44.364088    4460 scope.go:122] "RemoveContainer" containerID="a73559c973889aff2df223ab52e35a5f92a5f300f4069a71761fa1ff23333c6d"
Oct 31 22:17:44.364405 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:17:44.364381    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a73559c973889aff2df223ab52e35a5f92a5f300f4069a71761fa1ff23333c6d\": container with ID starting with a73559c973889aff2df223ab52e35a5f92a5f300f4069a71761fa1ff23333c6d not found: ID does not exist" containerID="a73559c973889aff2df223ab52e35a5f92a5f300f4069a71761fa1ff23333c6d"
Oct 31 22:17:44.364524 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:44.364408    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a73559c973889aff2df223ab52e35a5f92a5f300f4069a71761fa1ff23333c6d"} err="failed to get container status \"a73559c973889aff2df223ab52e35a5f92a5f300f4069a71761fa1ff23333c6d\": rpc error: code = NotFound desc = could not find container \"a73559c973889aff2df223ab52e35a5f92a5f300f4069a71761fa1ff23333c6d\": container with ID starting with a73559c973889aff2df223ab52e35a5f92a5f300f4069a71761fa1ff23333c6d not found: ID does not exist"
Oct 31 22:17:44.415660 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:44.415631    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-c16684-94dc7cbd7-gjj44"]
Oct 31 22:17:44.419450 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:44.419428    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-c16684-94dc7cbd7-gjj44"]
Oct 31 22:17:44.456003 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:44.455975    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-c16684-685b865b6c-qbtqr"]
Oct 31 22:17:44.462668 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:44.462645    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-c16684-685b865b6c-qbtqr"]
Oct 31 22:17:44.747036 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:44.746317    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:45.871969 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:45.871885    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:45.990673 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:45.990639    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:46.170366 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.170190    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:46.208500 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.208475    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-c34cbd-559d4ccccf-bxss4"
Oct 31 22:17:46.224978 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.224933    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="076218ca-4ba5-43dc-967e-fb30eda6e029" path="/var/lib/kubelet/pods/076218ca-4ba5-43dc-967e-fb30eda6e029/volumes"
Oct 31 22:17:46.225473 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.225452    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5ce9718e-7d7b-4fc4-b506-b25cd880efc3" path="/var/lib/kubelet/pods/5ce9718e-7d7b-4fc4-b506-b25cd880efc3/volumes"
Oct 31 22:17:46.282966 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.274269    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c34cbd-7dc696c947-v5wpt"
Oct 31 22:17:46.299966 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.298892    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e2e149ab-f40b-4d96-91df-b230dfae9e90-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2e149ab-f40b-4d96-91df-b230dfae9e90-plugins-config-volume\") pod \"e2e149ab-f40b-4d96-91df-b230dfae9e90\" (UID: \"e2e149ab-f40b-4d96-91df-b230dfae9e90\") "
Oct 31 22:17:46.299966 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.299082    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a22daeb7-8d30-4dcc-98be-e8f404ab1c42-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a22daeb7-8d30-4dcc-98be-e8f404ab1c42-plugins-config-volume\") pod \"a22daeb7-8d30-4dcc-98be-e8f404ab1c42\" (UID: \"a22daeb7-8d30-4dcc-98be-e8f404ab1c42\") "
Oct 31 22:17:46.299966 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.299175    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e2e149ab-f40b-4d96-91df-b230dfae9e90-kube-api-access-x4vqz\" (UniqueName: \"kubernetes.io/projected/e2e149ab-f40b-4d96-91df-b230dfae9e90-kube-api-access-x4vqz\") pod \"e2e149ab-f40b-4d96-91df-b230dfae9e90\" (UID: \"e2e149ab-f40b-4d96-91df-b230dfae9e90\") "
Oct 31 22:17:46.299966 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.299245    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a22daeb7-8d30-4dcc-98be-e8f404ab1c42-kube-api-access-88k8b\" (UniqueName: \"kubernetes.io/projected/a22daeb7-8d30-4dcc-98be-e8f404ab1c42-kube-api-access-88k8b\") pod \"a22daeb7-8d30-4dcc-98be-e8f404ab1c42\" (UID: \"a22daeb7-8d30-4dcc-98be-e8f404ab1c42\") "
Oct 31 22:17:46.305700 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.305395    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a22daeb7-8d30-4dcc-98be-e8f404ab1c42-plugins-config-volume" pod "a22daeb7-8d30-4dcc-98be-e8f404ab1c42" (UID: "a22daeb7-8d30-4dcc-98be-e8f404ab1c42"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:17:46.306719 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.306131    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2e149ab-f40b-4d96-91df-b230dfae9e90-plugins-config-volume" pod "e2e149ab-f40b-4d96-91df-b230dfae9e90" (UID: "e2e149ab-f40b-4d96-91df-b230dfae9e90"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:17:46.311095 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.309103    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2e149ab-f40b-4d96-91df-b230dfae9e90-kube-api-access-x4vqz" pod "e2e149ab-f40b-4d96-91df-b230dfae9e90" (UID: "e2e149ab-f40b-4d96-91df-b230dfae9e90"). InnerVolumeSpecName "kube-api-access-x4vqz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:17:46.313846 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.313821    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a22daeb7-8d30-4dcc-98be-e8f404ab1c42-kube-api-access-88k8b" pod "a22daeb7-8d30-4dcc-98be-e8f404ab1c42" (UID: "a22daeb7-8d30-4dcc-98be-e8f404ab1c42"). InnerVolumeSpecName "kube-api-access-88k8b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:17:46.365290 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.365254    4460 generic.go:356] "Generic (PLEG): container finished" podID="a22daeb7-8d30-4dcc-98be-e8f404ab1c42" containerID="821788ddfbf583e3a0219e06a757dda353f9ccd1dd26540ccc71c7a9f3be5a5b" exitCode=0
Oct 31 22:17:46.365499 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.365305    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c34cbd-7dc696c947-v5wpt"
Oct 31 22:17:46.365499 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.365374    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c34cbd-7dc696c947-v5wpt" event={"ID":"a22daeb7-8d30-4dcc-98be-e8f404ab1c42","Type":"ContainerDied","Data":"821788ddfbf583e3a0219e06a757dda353f9ccd1dd26540ccc71c7a9f3be5a5b"}
Oct 31 22:17:46.365499 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.365421    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-c34cbd-7dc696c947-v5wpt" event={"ID":"a22daeb7-8d30-4dcc-98be-e8f404ab1c42","Type":"ContainerDied","Data":"dfcd585ac12421b071bc91c1298e549231a910ee2bddeb07b0c27e54eec80a57"}
Oct 31 22:17:46.365499 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.365456    4460 scope.go:122] "RemoveContainer" containerID="821788ddfbf583e3a0219e06a757dda353f9ccd1dd26540ccc71c7a9f3be5a5b"
Oct 31 22:17:46.368212 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.368183    4460 generic.go:356] "Generic (PLEG): container finished" podID="e2e149ab-f40b-4d96-91df-b230dfae9e90" containerID="8f7e815b1a572b6f9ed988032d688cd0868ea44374f15808e8082a7d3d093825" exitCode=0
Oct 31 22:17:46.368350 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.368241    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-c34cbd-559d4ccccf-bxss4"
Oct 31 22:17:46.368350 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.368264    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-c34cbd-559d4ccccf-bxss4" event={"ID":"e2e149ab-f40b-4d96-91df-b230dfae9e90","Type":"ContainerDied","Data":"8f7e815b1a572b6f9ed988032d688cd0868ea44374f15808e8082a7d3d093825"}
Oct 31 22:17:46.368350 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.368310    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-c34cbd-559d4ccccf-bxss4" event={"ID":"e2e149ab-f40b-4d96-91df-b230dfae9e90","Type":"ContainerDied","Data":"7f2f6b84ef328713e88e6dc5e2845dbddf4afda629ab60949401f2a140b56b3d"}
Oct 31 22:17:46.379676 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.379653    4460 scope.go:122] "RemoveContainer" containerID="821788ddfbf583e3a0219e06a757dda353f9ccd1dd26540ccc71c7a9f3be5a5b"
Oct 31 22:17:46.380025 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:17:46.379999    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"821788ddfbf583e3a0219e06a757dda353f9ccd1dd26540ccc71c7a9f3be5a5b\": container with ID starting with 821788ddfbf583e3a0219e06a757dda353f9ccd1dd26540ccc71c7a9f3be5a5b not found: ID does not exist" containerID="821788ddfbf583e3a0219e06a757dda353f9ccd1dd26540ccc71c7a9f3be5a5b"
Oct 31 22:17:46.380113 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.380035    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"821788ddfbf583e3a0219e06a757dda353f9ccd1dd26540ccc71c7a9f3be5a5b"} err="failed to get container status \"821788ddfbf583e3a0219e06a757dda353f9ccd1dd26540ccc71c7a9f3be5a5b\": rpc error: code = NotFound desc = could not find container \"821788ddfbf583e3a0219e06a757dda353f9ccd1dd26540ccc71c7a9f3be5a5b\": container with ID starting with 821788ddfbf583e3a0219e06a757dda353f9ccd1dd26540ccc71c7a9f3be5a5b not found: ID does not exist"
Oct 31 22:17:46.380113 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.380058    4460 scope.go:122] "RemoveContainer" containerID="8f7e815b1a572b6f9ed988032d688cd0868ea44374f15808e8082a7d3d093825"
Oct 31 22:17:46.391183 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.391161    4460 scope.go:122] "RemoveContainer" containerID="8f7e815b1a572b6f9ed988032d688cd0868ea44374f15808e8082a7d3d093825"
Oct 31 22:17:46.391475 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:17:46.391454    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f7e815b1a572b6f9ed988032d688cd0868ea44374f15808e8082a7d3d093825\": container with ID starting with 8f7e815b1a572b6f9ed988032d688cd0868ea44374f15808e8082a7d3d093825 not found: ID does not exist" containerID="8f7e815b1a572b6f9ed988032d688cd0868ea44374f15808e8082a7d3d093825"
Oct 31 22:17:46.391568 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.391480    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f7e815b1a572b6f9ed988032d688cd0868ea44374f15808e8082a7d3d093825"} err="failed to get container status \"8f7e815b1a572b6f9ed988032d688cd0868ea44374f15808e8082a7d3d093825\": rpc error: code = NotFound desc = could not find container \"8f7e815b1a572b6f9ed988032d688cd0868ea44374f15808e8082a7d3d093825\": container with ID starting with 8f7e815b1a572b6f9ed988032d688cd0868ea44374f15808e8082a7d3d093825 not found: ID does not exist"
Oct 31 22:17:46.401469 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.401446    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88k8b\" (UniqueName: \"kubernetes.io/projected/a22daeb7-8d30-4dcc-98be-e8f404ab1c42-kube-api-access-88k8b\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:46.401562 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.401471    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2e149ab-f40b-4d96-91df-b230dfae9e90-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:46.401562 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.401486    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a22daeb7-8d30-4dcc-98be-e8f404ab1c42-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:46.401562 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.401499    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4vqz\" (UniqueName: \"kubernetes.io/projected/e2e149ab-f40b-4d96-91df-b230dfae9e90-kube-api-access-x4vqz\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:17:46.423518 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.423436    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-c34cbd-7dc696c947-v5wpt"]
Oct 31 22:17:46.427568 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.427546    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-c34cbd-7dc696c947-v5wpt"]
Oct 31 22:17:46.480769 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.480742    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-c34cbd-559d4ccccf-bxss4"]
Oct 31 22:17:46.485055 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.485035    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-c34cbd-559d4ccccf-bxss4"]
Oct 31 22:17:46.637763 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:46.637732    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:47.845022 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:47.844980    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-c08360-58948b75c9-k66nk" podUID="49303f39-9a79-4b1e-b8dc-12edad88fc96" containerName="payload-processing" containerID="cri-o://04a2df8b073f6013d67c391efee51e660d3533928ea88baf9d832ea7031d163b" gracePeriod=25
Oct 31 22:17:47.878556 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:47.878523    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c08360-8645d7ff59-s2tn2" podUID="327e9388-0750-420d-ac21-2ee5fb4224d4" containerName="payload-pre-processing" containerID="cri-o://c46c876c32c20c0de3a882e2489b5b8e4bb84b53ca4ac0d778cf288862564435" gracePeriod=25
Oct 31 22:17:48.225587 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:48.225511    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a22daeb7-8d30-4dcc-98be-e8f404ab1c42" path="/var/lib/kubelet/pods/a22daeb7-8d30-4dcc-98be-e8f404ab1c42/volumes"
Oct 31 22:17:48.225950 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:48.225928    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e2e149ab-f40b-4d96-91df-b230dfae9e90" path="/var/lib/kubelet/pods/e2e149ab-f40b-4d96-91df-b230dfae9e90/volumes"
Oct 31 22:17:48.583962 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:48.583885    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:48.915771 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:48.915690    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:49.040959 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:49.036583    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:50.113128 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:50.113087    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:50.604766 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:50.604729    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:50.720901 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:50.720864    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:50.828671 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:50.828635    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:51.101628 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:51.101592    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:51.264021 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:51.263988    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:51.379566 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:51.379479    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:51.453806 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:17:51.453774    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:17:52.548668 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:52.548633    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:54.278461 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:54.278425    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:54.583163 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:54.583081    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-inf-b-c08360-58948b75c9-k66nk" podUID="49303f39-9a79-4b1e-b8dc-12edad88fc96" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:17:55.592493 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:55.592447    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c08360-8645d7ff59-s2tn2" podUID="327e9388-0750-420d-ac21-2ee5fb4224d4" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:17:56.052240 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:56.052199    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:57.331337 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:57.331303    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:57.467773 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:57.467737    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:57.641601 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:57.641510    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:58.135745 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:58.135713    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:58.254377 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:58.254344    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:58.360809 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:58.360772    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:58.617469 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:58.617440    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:58.732580 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:58.732546    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:58.849685 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:58.849646    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:17:58.957198 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:58.957117    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"]
Oct 31 22:17:58.957378 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:58.957353    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df" podUID="e97b2260-7cb0-4f61-a862-25b9313beb18" containerName="istio-proxy" containerID="cri-o://6f9b7a3fa6046f4630e0994091add07ba46808e925339a939754db6e63472ffa" gracePeriod=30
Oct 31 22:17:58.967742 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:17:58.967709    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:00.224131 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:00.224094    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:01.476429 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:01.476395    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:01.996389 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:01.996350    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:03.713346 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:03.713310    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:04.218829 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.218805    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 22:18:04.277225 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.277200    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-credential-socket\") pod \"e97b2260-7cb0-4f61-a862-25b9313beb18\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") "
Oct 31 22:18:04.277350 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.277233    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-workload-socket\") pod \"e97b2260-7cb0-4f61-a862-25b9313beb18\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") "
Oct 31 22:18:04.277350 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.277305    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e97b2260-7cb0-4f61-a862-25b9313beb18-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e97b2260-7cb0-4f61-a862-25b9313beb18-istiod-ca-cert\") pod \"e97b2260-7cb0-4f61-a862-25b9313beb18\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") "
Oct 31 22:18:04.277478 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.277353    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-istio-data\") pod \"e97b2260-7cb0-4f61-a862-25b9313beb18\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") "
Oct 31 22:18:04.277478 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.277417    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e97b2260-7cb0-4f61-a862-25b9313beb18-istio-token\" (UniqueName: \"kubernetes.io/projected/e97b2260-7cb0-4f61-a862-25b9313beb18-istio-token\") pod \"e97b2260-7cb0-4f61-a862-25b9313beb18\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") "
Oct 31 22:18:04.277478 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.277471    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e97b2260-7cb0-4f61-a862-25b9313beb18-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e97b2260-7cb0-4f61-a862-25b9313beb18-istio-podinfo\") pod \"e97b2260-7cb0-4f61-a862-25b9313beb18\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") "
Oct 31 22:18:04.277652 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.277471    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-credential-socket" pod "e97b2260-7cb0-4f61-a862-25b9313beb18" (UID: "e97b2260-7cb0-4f61-a862-25b9313beb18"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:18:04.277652 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.277546    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-workload-certs\") pod \"e97b2260-7cb0-4f61-a862-25b9313beb18\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") "
Oct 31 22:18:04.277652 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.277563    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-workload-socket" pod "e97b2260-7cb0-4f61-a862-25b9313beb18" (UID: "e97b2260-7cb0-4f61-a862-25b9313beb18"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:18:04.277652 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.277602    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e97b2260-7cb0-4f61-a862-25b9313beb18-kube-api-access-lq6h7\" (UniqueName: \"kubernetes.io/projected/e97b2260-7cb0-4f61-a862-25b9313beb18-kube-api-access-lq6h7\") pod \"e97b2260-7cb0-4f61-a862-25b9313beb18\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") "
Oct 31 22:18:04.277883 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.277674    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-istio-envoy\") pod \"e97b2260-7cb0-4f61-a862-25b9313beb18\" (UID: \"e97b2260-7cb0-4f61-a862-25b9313beb18\") "
Oct 31 22:18:04.277883 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.277682    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-istio-data" pod "e97b2260-7cb0-4f61-a862-25b9313beb18" (UID: "e97b2260-7cb0-4f61-a862-25b9313beb18"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:18:04.277883 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.277729    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e97b2260-7cb0-4f61-a862-25b9313beb18-istiod-ca-cert" pod "e97b2260-7cb0-4f61-a862-25b9313beb18" (UID: "e97b2260-7cb0-4f61-a862-25b9313beb18"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:18:04.278080 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.277896    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-workload-certs" pod "e97b2260-7cb0-4f61-a862-25b9313beb18" (UID: "e97b2260-7cb0-4f61-a862-25b9313beb18"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:18:04.278589 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.278288    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:18:04.278746 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.278703    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:18:04.278746 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.278725    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:18:04.278746 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.278741    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e97b2260-7cb0-4f61-a862-25b9313beb18-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:18:04.279011 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.278755    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:18:04.281153 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.281107    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e97b2260-7cb0-4f61-a862-25b9313beb18-istio-token" pod "e97b2260-7cb0-4f61-a862-25b9313beb18" (UID: "e97b2260-7cb0-4f61-a862-25b9313beb18"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:18:04.281153 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.281149    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e97b2260-7cb0-4f61-a862-25b9313beb18-kube-api-access-lq6h7" pod "e97b2260-7cb0-4f61-a862-25b9313beb18" (UID: "e97b2260-7cb0-4f61-a862-25b9313beb18"). InnerVolumeSpecName "kube-api-access-lq6h7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:18:04.284915 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.281664    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-istio-envoy" pod "e97b2260-7cb0-4f61-a862-25b9313beb18" (UID: "e97b2260-7cb0-4f61-a862-25b9313beb18"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:18:04.284915 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.281915    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e97b2260-7cb0-4f61-a862-25b9313beb18-istio-podinfo" pod "e97b2260-7cb0-4f61-a862-25b9313beb18" (UID: "e97b2260-7cb0-4f61-a862-25b9313beb18"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 22:18:04.380333 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.380306    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lq6h7\" (UniqueName: \"kubernetes.io/projected/e97b2260-7cb0-4f61-a862-25b9313beb18-kube-api-access-lq6h7\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:18:04.380333 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.380332    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e97b2260-7cb0-4f61-a862-25b9313beb18-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:18:04.380505 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.380341    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e97b2260-7cb0-4f61-a862-25b9313beb18-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:18:04.380505 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.380349    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e97b2260-7cb0-4f61-a862-25b9313beb18-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:18:04.516292 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.516257    4460 generic.go:356] "Generic (PLEG): container finished" podID="e97b2260-7cb0-4f61-a862-25b9313beb18" containerID="6f9b7a3fa6046f4630e0994091add07ba46808e925339a939754db6e63472ffa" exitCode=0
Oct 31 22:18:04.516458 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.516312    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"
Oct 31 22:18:04.516458 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.516343    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df" event={"ID":"e97b2260-7cb0-4f61-a862-25b9313beb18","Type":"ContainerDied","Data":"6f9b7a3fa6046f4630e0994091add07ba46808e925339a939754db6e63472ffa"}
Oct 31 22:18:04.516458 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.516396    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df" event={"ID":"e97b2260-7cb0-4f61-a862-25b9313beb18","Type":"ContainerDied","Data":"9245a5af7a9295c78eb0b3362d36a75b74ac41ac265c22be051cfe0a35b9974c"}
Oct 31 22:18:04.516458 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.516421    4460 scope.go:122] "RemoveContainer" containerID="6f9b7a3fa6046f4630e0994091add07ba46808e925339a939754db6e63472ffa"
Oct 31 22:18:04.527525 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.527505    4460 scope.go:122] "RemoveContainer" containerID="6f9b7a3fa6046f4630e0994091add07ba46808e925339a939754db6e63472ffa"
Oct 31 22:18:04.527812 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:18:04.527791    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f9b7a3fa6046f4630e0994091add07ba46808e925339a939754db6e63472ffa\": container with ID starting with 6f9b7a3fa6046f4630e0994091add07ba46808e925339a939754db6e63472ffa not found: ID does not exist" containerID="6f9b7a3fa6046f4630e0994091add07ba46808e925339a939754db6e63472ffa"
Oct 31 22:18:04.527877 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.527822    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f9b7a3fa6046f4630e0994091add07ba46808e925339a939754db6e63472ffa"} err="failed to get container status \"6f9b7a3fa6046f4630e0994091add07ba46808e925339a939754db6e63472ffa\": rpc error: code = NotFound desc = could not find container \"6f9b7a3fa6046f4630e0994091add07ba46808e925339a939754db6e63472ffa\": container with ID starting with 6f9b7a3fa6046f4630e0994091add07ba46808e925339a939754db6e63472ffa not found: ID does not exist"
Oct 31 22:18:04.583067 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.583028    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-inf-b-c08360-58948b75c9-k66nk" podUID="49303f39-9a79-4b1e-b8dc-12edad88fc96" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:18:04.586204 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.586177    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"]
Oct 31 22:18:04.599427 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:04.599393    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-c16684-openshift-default-77c759c989-2q2df"]
Oct 31 22:18:05.521741 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:05.521702    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:05.593533 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:05.593490    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c08360-8645d7ff59-s2tn2" podUID="327e9388-0750-420d-ac21-2ee5fb4224d4" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:18:05.747142 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:05.747099    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:05.855640 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:05.855545    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:06.053794 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:06.053765    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:06.225628 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:06.225549    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e97b2260-7cb0-4f61-a862-25b9313beb18" path="/var/lib/kubelet/pods/e97b2260-7cb0-4f61-a862-25b9313beb18/volumes"
Oct 31 22:18:07.215627 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:07.215594    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 22:18:07.310054 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:07.310022    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:08.162008 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:08.161980    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c08360-58948b75c9-k66nk"
Oct 31 22:18:08.187549 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:08.187519    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c08360-8645d7ff59-s2tn2"
Oct 31 22:18:08.233030 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:08.232997    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/49303f39-9a79-4b1e-b8dc-12edad88fc96-kube-api-access-9g8d2\" (UniqueName: \"kubernetes.io/projected/49303f39-9a79-4b1e-b8dc-12edad88fc96-kube-api-access-9g8d2\") pod \"49303f39-9a79-4b1e-b8dc-12edad88fc96\" (UID: \"49303f39-9a79-4b1e-b8dc-12edad88fc96\") "
Oct 31 22:18:08.233529 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:08.233062    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/49303f39-9a79-4b1e-b8dc-12edad88fc96-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49303f39-9a79-4b1e-b8dc-12edad88fc96-plugins-config-volume\") pod \"49303f39-9a79-4b1e-b8dc-12edad88fc96\" (UID: \"49303f39-9a79-4b1e-b8dc-12edad88fc96\") "
Oct 31 22:18:08.233529 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:08.233445    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49303f39-9a79-4b1e-b8dc-12edad88fc96-plugins-config-volume" pod "49303f39-9a79-4b1e-b8dc-12edad88fc96" (UID: "49303f39-9a79-4b1e-b8dc-12edad88fc96"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:18:08.233736 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:08.233711    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49303f39-9a79-4b1e-b8dc-12edad88fc96-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:18:08.235490 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:08.235461    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49303f39-9a79-4b1e-b8dc-12edad88fc96-kube-api-access-9g8d2" pod "49303f39-9a79-4b1e-b8dc-12edad88fc96" (UID: "49303f39-9a79-4b1e-b8dc-12edad88fc96"). InnerVolumeSpecName "kube-api-access-9g8d2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:18:08.334285 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:08.334254    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/327e9388-0750-420d-ac21-2ee5fb4224d4-kube-api-access-lj4qz\" (UniqueName: \"kubernetes.io/projected/327e9388-0750-420d-ac21-2ee5fb4224d4-kube-api-access-lj4qz\") pod \"327e9388-0750-420d-ac21-2ee5fb4224d4\" (UID: \"327e9388-0750-420d-ac21-2ee5fb4224d4\") "
Oct 31 22:18:08.334493 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:08.334477    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/327e9388-0750-420d-ac21-2ee5fb4224d4-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/327e9388-0750-420d-ac21-2ee5fb4224d4-plugins-config-volume\") pod \"327e9388-0750-420d-ac21-2ee5fb4224d4\" (UID: \"327e9388-0750-420d-ac21-2ee5fb4224d4\") "
Oct 31 22:18:08.334950 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:08.334918    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9g8d2\" (UniqueName: \"kubernetes.io/projected/49303f39-9a79-4b1e-b8dc-12edad88fc96-kube-api-access-9g8d2\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:18:08.334950 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:08.334910    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/327e9388-0750-420d-ac21-2ee5fb4224d4-plugins-config-volume" pod "327e9388-0750-420d-ac21-2ee5fb4224d4" (UID: "327e9388-0750-420d-ac21-2ee5fb4224d4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:18:08.336520 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:08.336496    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/327e9388-0750-420d-ac21-2ee5fb4224d4-kube-api-access-lj4qz" pod "327e9388-0750-420d-ac21-2ee5fb4224d4" (UID: "327e9388-0750-420d-ac21-2ee5fb4224d4"). InnerVolumeSpecName "kube-api-access-lj4qz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:18:08.436117 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:08.436055    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/327e9388-0750-420d-ac21-2ee5fb4224d4-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:18:08.436117 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:08.436078    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lj4qz\" (UniqueName: \"kubernetes.io/projected/327e9388-0750-420d-ac21-2ee5fb4224d4-kube-api-access-lj4qz\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:18:08.544001 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:08.543972    4460 generic.go:356] "Generic (PLEG): container finished" podID="327e9388-0750-420d-ac21-2ee5fb4224d4" containerID="c46c876c32c20c0de3a882e2489b5b8e4bb84b53ca4ac0d778cf288862564435" exitCode=0
Oct 31 22:18:08.544139 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:08.544015    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c08360-8645d7ff59-s2tn2"
Oct 31 22:18:08.544139 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:08.544054    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c08360-8645d7ff59-s2tn2" event={"ID":"327e9388-0750-420d-ac21-2ee5fb4224d4","Type":"ContainerDied","Data":"c46c876c32c20c0de3a882e2489b5b8e4bb84b53ca4ac0d778cf288862564435"}
Oct 31 22:18:08.544139 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:08.544118    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c08360-8645d7ff59-s2tn2" event={"ID":"327e9388-0750-420d-ac21-2ee5fb4224d4","Type":"ContainerDied","Data":"226890eb0520a541e04584bba263d92472af3fa6eba330e1ee072128104d20e1"}
Oct 31 22:18:08.544271 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:08.544155    4460 scope.go:122] "RemoveContainer" containerID="c46c876c32c20c0de3a882e2489b5b8e4bb84b53ca4ac0d778cf288862564435"
Oct 31 22:18:08.545550 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:08.545522    4460 generic.go:356] "Generic (PLEG): container finished" podID="49303f39-9a79-4b1e-b8dc-12edad88fc96" containerID="04a2df8b073f6013d67c391efee51e660d3533928ea88baf9d832ea7031d163b" exitCode=0
Oct 31 22:18:08.545675 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:08.545571    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c08360-58948b75c9-k66nk"
Oct 31 22:18:08.545675 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:08.545612    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c08360-58948b75c9-k66nk" event={"ID":"49303f39-9a79-4b1e-b8dc-12edad88fc96","Type":"ContainerDied","Data":"04a2df8b073f6013d67c391efee51e660d3533928ea88baf9d832ea7031d163b"}
Oct 31 22:18:08.545675 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:08.545655    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c08360-58948b75c9-k66nk" event={"ID":"49303f39-9a79-4b1e-b8dc-12edad88fc96","Type":"ContainerDied","Data":"103c0a5aa917ad41a28d1eb3f02ea5a028e07db6c81396f9f7b9f88e86d5492b"}
Oct 31 22:18:08.555172 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:08.555047    4460 scope.go:122] "RemoveContainer" containerID="c46c876c32c20c0de3a882e2489b5b8e4bb84b53ca4ac0d778cf288862564435"
Oct 31 22:18:08.555395 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:18:08.555372    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c46c876c32c20c0de3a882e2489b5b8e4bb84b53ca4ac0d778cf288862564435\": container with ID starting with c46c876c32c20c0de3a882e2489b5b8e4bb84b53ca4ac0d778cf288862564435 not found: ID does not exist" containerID="c46c876c32c20c0de3a882e2489b5b8e4bb84b53ca4ac0d778cf288862564435"
Oct 31 22:18:08.555460 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:08.555414    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c46c876c32c20c0de3a882e2489b5b8e4bb84b53ca4ac0d778cf288862564435"} err="failed to get container status \"c46c876c32c20c0de3a882e2489b5b8e4bb84b53ca4ac0d778cf288862564435\": rpc error: code = NotFound desc = could not find container \"c46c876c32c20c0de3a882e2489b5b8e4bb84b53ca4ac0d778cf288862564435\": container with ID starting with c46c876c32c20c0de3a882e2489b5b8e4bb84b53ca4ac0d778cf288862564435 not found: ID does not exist"
Oct 31 22:18:08.555460 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:08.555436    4460 scope.go:122] "RemoveContainer" containerID="04a2df8b073f6013d67c391efee51e660d3533928ea88baf9d832ea7031d163b"
Oct 31 22:18:08.564999 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:08.564974    4460 scope.go:122] "RemoveContainer" containerID="04a2df8b073f6013d67c391efee51e660d3533928ea88baf9d832ea7031d163b"
Oct 31 22:18:08.565290 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:18:08.565264    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04a2df8b073f6013d67c391efee51e660d3533928ea88baf9d832ea7031d163b\": container with ID starting with 04a2df8b073f6013d67c391efee51e660d3533928ea88baf9d832ea7031d163b not found: ID does not exist" containerID="04a2df8b073f6013d67c391efee51e660d3533928ea88baf9d832ea7031d163b"
Oct 31 22:18:08.565361 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:08.565299    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04a2df8b073f6013d67c391efee51e660d3533928ea88baf9d832ea7031d163b"} err="failed to get container status \"04a2df8b073f6013d67c391efee51e660d3533928ea88baf9d832ea7031d163b\": rpc error: code = NotFound desc = could not find container \"04a2df8b073f6013d67c391efee51e660d3533928ea88baf9d832ea7031d163b\": container with ID starting with 04a2df8b073f6013d67c391efee51e660d3533928ea88baf9d832ea7031d163b not found: ID does not exist"
Oct 31 22:18:08.606976 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:08.604116    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c08360-58948b75c9-k66nk"]
Oct 31 22:18:08.608222 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:08.608198    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c08360-58948b75c9-k66nk"]
Oct 31 22:18:08.654167 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:08.654126    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-c08360-8645d7ff59-s2tn2"]
Oct 31 22:18:08.657522 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:08.657497    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-c08360-8645d7ff59-s2tn2"]
Oct 31 22:18:09.144877 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:09.144782    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:10.225219 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:10.225179    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="327e9388-0750-420d-ac21-2ee5fb4224d4" path="/var/lib/kubelet/pods/327e9388-0750-420d-ac21-2ee5fb4224d4/volumes"
Oct 31 22:18:10.225813 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:10.225788    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="49303f39-9a79-4b1e-b8dc-12edad88fc96" path="/var/lib/kubelet/pods/49303f39-9a79-4b1e-b8dc-12edad88fc96/volumes"
Oct 31 22:18:10.735955 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:10.735916    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:12.323528 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:12.323489    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:13.994139 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:13.994052    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:15.519228 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:15.519192    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:16.751186 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:16.751146    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45"]
Oct 31 22:18:16.751641 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:16.751408    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45" podUID="93b36b86-cb3d-41a0-b3b8-526a57e417da" containerName="maas-api" containerID="cri-o://87d7dde05aa05bd81e466f13fc74917eb0d0b9b85f695ce735e8a0a2834f3bea" gracePeriod=30
Oct 31 22:18:16.813984 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:16.813913    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:17.039151 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:17.039121    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45"
Oct 31 22:18:17.072705 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:17.072669    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-696f9a-67f544df69-gb7nm"]
Oct 31 22:18:17.117761 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:17.117715    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-696f9a-fc885d6c9-pnrxl"]
Oct 31 22:18:17.139671 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:17.139644    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/93b36b86-cb3d-41a0-b3b8-526a57e417da-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93b36b86-cb3d-41a0-b3b8-526a57e417da-maas-api-tls\") pod \"93b36b86-cb3d-41a0-b3b8-526a57e417da\" (UID: \"93b36b86-cb3d-41a0-b3b8-526a57e417da\") "
Oct 31 22:18:17.139835 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:17.139714    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/93b36b86-cb3d-41a0-b3b8-526a57e417da-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93b36b86-cb3d-41a0-b3b8-526a57e417da-openshift-service-ca\") pod \"93b36b86-cb3d-41a0-b3b8-526a57e417da\" (UID: \"93b36b86-cb3d-41a0-b3b8-526a57e417da\") "
Oct 31 22:18:17.139887 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:17.139866    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/93b36b86-cb3d-41a0-b3b8-526a57e417da-metrics-certs\" (UniqueName: \"kubernetes.io/secret/93b36b86-cb3d-41a0-b3b8-526a57e417da-metrics-certs\") pod \"93b36b86-cb3d-41a0-b3b8-526a57e417da\" (UID: \"93b36b86-cb3d-41a0-b3b8-526a57e417da\") "
Oct 31 22:18:17.139960 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:17.139919    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93b36b86-cb3d-41a0-b3b8-526a57e417da-kube-api-access-qhv9p\" (UniqueName: \"kubernetes.io/projected/93b36b86-cb3d-41a0-b3b8-526a57e417da-kube-api-access-qhv9p\") pod \"93b36b86-cb3d-41a0-b3b8-526a57e417da\" (UID: \"93b36b86-cb3d-41a0-b3b8-526a57e417da\") "
Oct 31 22:18:17.140103 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:17.140077    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93b36b86-cb3d-41a0-b3b8-526a57e417da-openshift-service-ca" pod "93b36b86-cb3d-41a0-b3b8-526a57e417da" (UID: "93b36b86-cb3d-41a0-b3b8-526a57e417da"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:18:17.140430 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:17.140408    4460 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93b36b86-cb3d-41a0-b3b8-526a57e417da-openshift-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:18:17.142002 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:17.141975    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93b36b86-cb3d-41a0-b3b8-526a57e417da-kube-api-access-qhv9p" pod "93b36b86-cb3d-41a0-b3b8-526a57e417da" (UID: "93b36b86-cb3d-41a0-b3b8-526a57e417da"). InnerVolumeSpecName "kube-api-access-qhv9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:18:17.142106 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:17.141998    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93b36b86-cb3d-41a0-b3b8-526a57e417da-metrics-certs" pod "93b36b86-cb3d-41a0-b3b8-526a57e417da" (UID: "93b36b86-cb3d-41a0-b3b8-526a57e417da"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 22:18:17.142106 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:17.142076    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93b36b86-cb3d-41a0-b3b8-526a57e417da-maas-api-tls" pod "93b36b86-cb3d-41a0-b3b8-526a57e417da" (UID: "93b36b86-cb3d-41a0-b3b8-526a57e417da"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 22:18:17.225543 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:17.225511    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:17.242128 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:17.242101    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93b36b86-cb3d-41a0-b3b8-526a57e417da-maas-api-tls\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:18:17.242128 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:17.242131    4460 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93b36b86-cb3d-41a0-b3b8-526a57e417da-metrics-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:18:17.242309 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:17.242145    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhv9p\" (UniqueName: \"kubernetes.io/projected/93b36b86-cb3d-41a0-b3b8-526a57e417da-kube-api-access-qhv9p\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:18:17.353043 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:17.352933    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:17.473266 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:17.473226    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:17.608218 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:17.608129    4460 generic.go:356] "Generic (PLEG): container finished" podID="93b36b86-cb3d-41a0-b3b8-526a57e417da" containerID="87d7dde05aa05bd81e466f13fc74917eb0d0b9b85f695ce735e8a0a2834f3bea" exitCode=0
Oct 31 22:18:17.608218 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:17.608179    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45"
Oct 31 22:18:17.608474 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:17.608218    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45" event={"ID":"93b36b86-cb3d-41a0-b3b8-526a57e417da","Type":"ContainerDied","Data":"87d7dde05aa05bd81e466f13fc74917eb0d0b9b85f695ce735e8a0a2834f3bea"}
Oct 31 22:18:17.608474 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:17.608288    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45" event={"ID":"93b36b86-cb3d-41a0-b3b8-526a57e417da","Type":"ContainerDied","Data":"34fe52054f069652ba0096935317f530463c822d42849d9b11006f4791bc20aa"}
Oct 31 22:18:17.608474 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:17.608315    4460 scope.go:122] "RemoveContainer" containerID="87d7dde05aa05bd81e466f13fc74917eb0d0b9b85f695ce735e8a0a2834f3bea"
Oct 31 22:18:17.619715 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:17.619697    4460 scope.go:122] "RemoveContainer" containerID="87d7dde05aa05bd81e466f13fc74917eb0d0b9b85f695ce735e8a0a2834f3bea"
Oct 31 22:18:17.620033 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:18:17.620010    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87d7dde05aa05bd81e466f13fc74917eb0d0b9b85f695ce735e8a0a2834f3bea\": container with ID starting with 87d7dde05aa05bd81e466f13fc74917eb0d0b9b85f695ce735e8a0a2834f3bea not found: ID does not exist" containerID="87d7dde05aa05bd81e466f13fc74917eb0d0b9b85f695ce735e8a0a2834f3bea"
Oct 31 22:18:17.620086 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:17.620041    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87d7dde05aa05bd81e466f13fc74917eb0d0b9b85f695ce735e8a0a2834f3bea"} err="failed to get container status \"87d7dde05aa05bd81e466f13fc74917eb0d0b9b85f695ce735e8a0a2834f3bea\": rpc error: code = NotFound desc = could not find container \"87d7dde05aa05bd81e466f13fc74917eb0d0b9b85f695ce735e8a0a2834f3bea\": container with ID starting with 87d7dde05aa05bd81e466f13fc74917eb0d0b9b85f695ce735e8a0a2834f3bea not found: ID does not exist"
Oct 31 22:18:17.665673 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:17.665649    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45"]
Oct 31 22:18:17.679203 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:17.679179    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-f1e791-5b558c6694-55c45"]
Oct 31 22:18:17.883882 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:17.883799    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:18.225156 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:18.225084    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="93b36b86-cb3d-41a0-b3b8-526a57e417da" path="/var/lib/kubelet/pods/93b36b86-cb3d-41a0-b3b8-526a57e417da/volumes"
Oct 31 22:18:19.128352 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:19.128309    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:19.242159 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:19.242127    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:20.982185 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:20.982141    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:21.452571 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:18:21.452535    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:18:22.073200 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:22.073156    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-696f9a-67f544df69-gb7nm" podUID="869e1829-ba34-4ed7-bd97-37aa003a0d0f" containerName="payload-processing" containerID="cri-o://6abf9618c0148ceb6158fe31ae2e799e97136ead6f17693dd0fb2de1ca58fa70" gracePeriod=25
Oct 31 22:18:22.118326 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:22.118288    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-696f9a-fc885d6c9-pnrxl" podUID="de079877-5ceb-4370-b8f6-9f6644de83b3" containerName="payload-pre-processing" containerID="cri-o://aad578a8ba1d909be721ccd1999fd710e09e470e5f24ad2bb8035af516118252" gracePeriod=25
Oct 31 22:18:22.644074 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:22.644037    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:24.242831 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:24.242783    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:24.362340 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:24.362299    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:24.472404 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:24.472367    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:24.538574 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:24.538533    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"]
Oct 31 22:18:24.538858 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:24.538827    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm" podUID="9a09d750-0cc0-430b-a2d6-6490cf9f1361" containerName="istio-proxy" containerID="cri-o://9d2ccabc9072b0943ccae28d396621786180a78ffdc1dfec174c660f97615d8f" gracePeriod=30
Oct 31 22:18:24.589504 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:24.589468    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:26.269266 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:26.269230    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:26.796781 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:26.796739    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-696f9a-fc885d6c9-pnrxl" podUID="de079877-5ceb-4370-b8f6-9f6644de83b3" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:18:26.796988 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:26.796754    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-696f9a-67f544df69-gb7nm" podUID="869e1829-ba34-4ed7-bd97-37aa003a0d0f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:18:27.885599 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:27.885569    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 22:18:28.131449 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:28.131410    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:29.721590 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:29.721552    4460 generic.go:356] "Generic (PLEG): container finished" podID="9a09d750-0cc0-430b-a2d6-6490cf9f1361" containerID="9d2ccabc9072b0943ccae28d396621786180a78ffdc1dfec174c660f97615d8f" exitCode=0
Oct 31 22:18:29.722010 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:29.721631    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm" event={"ID":"9a09d750-0cc0-430b-a2d6-6490cf9f1361","Type":"ContainerDied","Data":"9d2ccabc9072b0943ccae28d396621786180a78ffdc1dfec174c660f97615d8f"}
Oct 31 22:18:29.722010 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:29.721694    4460 scope.go:122] "RemoveContainer" containerID="3042a385bef9cf71215d9cac8aa2098c635c2f82effbb7d7b18e08bb1726d6ec"
Oct 31 22:18:29.832957 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:29.832878    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 22:18:29.939324 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:29.939285    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istio-token\" (UniqueName: \"kubernetes.io/projected/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istio-token\") pod \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") "
Oct 31 22:18:29.939506 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:29.939341    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-workload-socket\") pod \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") "
Oct 31 22:18:29.939506 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:29.939385    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istio-podinfo\") pod \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") "
Oct 31 22:18:29.939506 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:29.939457    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istio-envoy\") pod \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") "
Oct 31 22:18:29.939506 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:29.939497    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istiod-ca-cert\") pod \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") "
Oct 31 22:18:29.939746 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:29.939631    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-credential-socket\") pod \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") "
Oct 31 22:18:29.939746 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:29.939669    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istio-data\") pod \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") "
Oct 31 22:18:29.939875 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:29.939779    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-workload-socket" pod "9a09d750-0cc0-430b-a2d6-6490cf9f1361" (UID: "9a09d750-0cc0-430b-a2d6-6490cf9f1361"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:18:29.939875 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:29.939828    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9a09d750-0cc0-430b-a2d6-6490cf9f1361-kube-api-access-f459v\" (UniqueName: \"kubernetes.io/projected/9a09d750-0cc0-430b-a2d6-6490cf9f1361-kube-api-access-f459v\") pod \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") "
Oct 31 22:18:29.939875 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:29.939860    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-workload-certs\") pod \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\" (UID: \"9a09d750-0cc0-430b-a2d6-6490cf9f1361\") "
Oct 31 22:18:29.940085 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:29.940062    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-credential-socket" pod "9a09d750-0cc0-430b-a2d6-6490cf9f1361" (UID: "9a09d750-0cc0-430b-a2d6-6490cf9f1361"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:18:29.940303 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:29.940170    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istio-data" pod "9a09d750-0cc0-430b-a2d6-6490cf9f1361" (UID: "9a09d750-0cc0-430b-a2d6-6490cf9f1361"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:18:29.940303 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:29.940278    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-workload-certs" pod "9a09d750-0cc0-430b-a2d6-6490cf9f1361" (UID: "9a09d750-0cc0-430b-a2d6-6490cf9f1361"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:18:29.940548 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:29.940527    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:18:29.940625 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:29.940554    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:18:29.940625 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:29.940571    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:18:29.940625 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:29.940584    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:18:29.940913 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:29.940879    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istiod-ca-cert" pod "9a09d750-0cc0-430b-a2d6-6490cf9f1361" (UID: "9a09d750-0cc0-430b-a2d6-6490cf9f1361"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:18:29.942323 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:29.942282    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istio-podinfo" pod "9a09d750-0cc0-430b-a2d6-6490cf9f1361" (UID: "9a09d750-0cc0-430b-a2d6-6490cf9f1361"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 22:18:29.942496 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:29.942469    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a09d750-0cc0-430b-a2d6-6490cf9f1361-kube-api-access-f459v" pod "9a09d750-0cc0-430b-a2d6-6490cf9f1361" (UID: "9a09d750-0cc0-430b-a2d6-6490cf9f1361"). InnerVolumeSpecName "kube-api-access-f459v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:18:29.942593 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:29.942577    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istio-envoy" pod "9a09d750-0cc0-430b-a2d6-6490cf9f1361" (UID: "9a09d750-0cc0-430b-a2d6-6490cf9f1361"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:18:29.942669 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:29.942592    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istio-token" pod "9a09d750-0cc0-430b-a2d6-6490cf9f1361" (UID: "9a09d750-0cc0-430b-a2d6-6490cf9f1361"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:18:29.949828 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:29.949800    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:30.042340 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:30.042296    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:18:30.042340 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:30.042334    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:18:30.042340 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:30.042344    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f459v\" (UniqueName: \"kubernetes.io/projected/9a09d750-0cc0-430b-a2d6-6490cf9f1361-kube-api-access-f459v\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:18:30.042605 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:30.042354    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:18:30.042605 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:30.042363    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a09d750-0cc0-430b-a2d6-6490cf9f1361-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:18:30.729116 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:30.729081    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm" event={"ID":"9a09d750-0cc0-430b-a2d6-6490cf9f1361","Type":"ContainerDied","Data":"3d1bd55417f1a096aa31cd8038d8dacdb32756b5f19b5ff1cef1490de44d8762"}
Oct 31 22:18:30.729602 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:30.729164    4460 scope.go:122] "RemoveContainer" containerID="9d2ccabc9072b0943ccae28d396621786180a78ffdc1dfec174c660f97615d8f"
Oct 31 22:18:30.729602 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:30.729096    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"
Oct 31 22:18:30.801430 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:30.801396    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"]
Oct 31 22:18:30.804188 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:30.804151    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-f1e791-openshift-default-85667f7765-4wkgm"]
Oct 31 22:18:31.459294 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:31.459256    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:31.918557 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:31.918355    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:32.031810 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:32.031770    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:32.145235 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:32.145196    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:32.242374 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:32.242276    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9a09d750-0cc0-430b-a2d6-6490cf9f1361" path="/var/lib/kubelet/pods/9a09d750-0cc0-430b-a2d6-6490cf9f1361/volumes"
Oct 31 22:18:32.254142 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:32.254106    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:32.650382 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:32.650344    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:34.023677 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:34.023641    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:35.710209 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:35.710171    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:35.819406 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:35.819372    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:35.935278 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:35.935238    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:36.797304 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:36.797267    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-696f9a-fc885d6c9-pnrxl" podUID="de079877-5ceb-4370-b8f6-9f6644de83b3" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:18:36.797845 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:36.797261    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-696f9a-67f544df69-gb7nm" podUID="869e1829-ba34-4ed7-bd97-37aa003a0d0f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:18:37.654837 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:37.654797    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:38.969993 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:38.969925    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:39.077450 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:39.077417    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:39.304360 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:39.304320    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:39.443427 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:39.443393    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:41.333055 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:41.332997    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:42.380427 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.380400    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-696f9a-67f544df69-gb7nm"
Oct 31 22:18:42.421530 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.421503    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-696f9a-fc885d6c9-pnrxl"
Oct 31 22:18:42.458423 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.458387    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/869e1829-ba34-4ed7-bd97-37aa003a0d0f-kube-api-access-mwwtz\" (UniqueName: \"kubernetes.io/projected/869e1829-ba34-4ed7-bd97-37aa003a0d0f-kube-api-access-mwwtz\") pod \"869e1829-ba34-4ed7-bd97-37aa003a0d0f\" (UID: \"869e1829-ba34-4ed7-bd97-37aa003a0d0f\") "
Oct 31 22:18:42.458673 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.458515    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/869e1829-ba34-4ed7-bd97-37aa003a0d0f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/869e1829-ba34-4ed7-bd97-37aa003a0d0f-plugins-config-volume\") pod \"869e1829-ba34-4ed7-bd97-37aa003a0d0f\" (UID: \"869e1829-ba34-4ed7-bd97-37aa003a0d0f\") "
Oct 31 22:18:42.458990 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.458959    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/869e1829-ba34-4ed7-bd97-37aa003a0d0f-plugins-config-volume" pod "869e1829-ba34-4ed7-bd97-37aa003a0d0f" (UID: "869e1829-ba34-4ed7-bd97-37aa003a0d0f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:18:42.459146 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.459123    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/869e1829-ba34-4ed7-bd97-37aa003a0d0f-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:18:42.460534 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.460511    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/869e1829-ba34-4ed7-bd97-37aa003a0d0f-kube-api-access-mwwtz" pod "869e1829-ba34-4ed7-bd97-37aa003a0d0f" (UID: "869e1829-ba34-4ed7-bd97-37aa003a0d0f"). InnerVolumeSpecName "kube-api-access-mwwtz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:18:42.560367 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.560320    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/de079877-5ceb-4370-b8f6-9f6644de83b3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de079877-5ceb-4370-b8f6-9f6644de83b3-plugins-config-volume\") pod \"de079877-5ceb-4370-b8f6-9f6644de83b3\" (UID: \"de079877-5ceb-4370-b8f6-9f6644de83b3\") "
Oct 31 22:18:42.560574 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.560428    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/de079877-5ceb-4370-b8f6-9f6644de83b3-kube-api-access-mxmbl\" (UniqueName: \"kubernetes.io/projected/de079877-5ceb-4370-b8f6-9f6644de83b3-kube-api-access-mxmbl\") pod \"de079877-5ceb-4370-b8f6-9f6644de83b3\" (UID: \"de079877-5ceb-4370-b8f6-9f6644de83b3\") "
Oct 31 22:18:42.560737 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.560713    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de079877-5ceb-4370-b8f6-9f6644de83b3-plugins-config-volume" pod "de079877-5ceb-4370-b8f6-9f6644de83b3" (UID: "de079877-5ceb-4370-b8f6-9f6644de83b3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:18:42.560829 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.560817    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de079877-5ceb-4370-b8f6-9f6644de83b3-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:18:42.560870 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.560833    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwwtz\" (UniqueName: \"kubernetes.io/projected/869e1829-ba34-4ed7-bd97-37aa003a0d0f-kube-api-access-mwwtz\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:18:42.562472 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.562451    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de079877-5ceb-4370-b8f6-9f6644de83b3-kube-api-access-mxmbl" pod "de079877-5ceb-4370-b8f6-9f6644de83b3" (UID: "de079877-5ceb-4370-b8f6-9f6644de83b3"). InnerVolumeSpecName "kube-api-access-mxmbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:18:42.662010 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.661915    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxmbl\" (UniqueName: \"kubernetes.io/projected/de079877-5ceb-4370-b8f6-9f6644de83b3-kube-api-access-mxmbl\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:18:42.788652 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.788608    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-1b0cce-7996cb486f-vpwrv"]
Oct 31 22:18:42.810710 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.810675    4460 generic.go:356] "Generic (PLEG): container finished" podID="de079877-5ceb-4370-b8f6-9f6644de83b3" containerID="aad578a8ba1d909be721ccd1999fd710e09e470e5f24ad2bb8035af516118252" exitCode=0
Oct 31 22:18:42.810906 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.810722    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-696f9a-fc885d6c9-pnrxl"
Oct 31 22:18:42.810906 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.810761    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-696f9a-fc885d6c9-pnrxl" event={"ID":"de079877-5ceb-4370-b8f6-9f6644de83b3","Type":"ContainerDied","Data":"aad578a8ba1d909be721ccd1999fd710e09e470e5f24ad2bb8035af516118252"}
Oct 31 22:18:42.810906 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.810817    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-696f9a-fc885d6c9-pnrxl" event={"ID":"de079877-5ceb-4370-b8f6-9f6644de83b3","Type":"ContainerDied","Data":"80dafd19e1fea0ec25e0e80c17199054fb634ff04ec38f9f029c2a57d393af63"}
Oct 31 22:18:42.810906 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.810841    4460 scope.go:122] "RemoveContainer" containerID="aad578a8ba1d909be721ccd1999fd710e09e470e5f24ad2bb8035af516118252"
Oct 31 22:18:42.812338 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.812314    4460 generic.go:356] "Generic (PLEG): container finished" podID="869e1829-ba34-4ed7-bd97-37aa003a0d0f" containerID="6abf9618c0148ceb6158fe31ae2e799e97136ead6f17693dd0fb2de1ca58fa70" exitCode=0
Oct 31 22:18:42.812465 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.812366    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-696f9a-67f544df69-gb7nm"
Oct 31 22:18:42.812465 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.812418    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-696f9a-67f544df69-gb7nm" event={"ID":"869e1829-ba34-4ed7-bd97-37aa003a0d0f","Type":"ContainerDied","Data":"6abf9618c0148ceb6158fe31ae2e799e97136ead6f17693dd0fb2de1ca58fa70"}
Oct 31 22:18:42.812465 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.812462    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-696f9a-67f544df69-gb7nm" event={"ID":"869e1829-ba34-4ed7-bd97-37aa003a0d0f","Type":"ContainerDied","Data":"0898fdfca23d8faafad7eb7e2dc8d49ccbead516250fa69da50e9230383ed389"}
Oct 31 22:18:42.823001 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.822850    4460 scope.go:122] "RemoveContainer" containerID="aad578a8ba1d909be721ccd1999fd710e09e470e5f24ad2bb8035af516118252"
Oct 31 22:18:42.823337 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:18:42.823306    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aad578a8ba1d909be721ccd1999fd710e09e470e5f24ad2bb8035af516118252\": container with ID starting with aad578a8ba1d909be721ccd1999fd710e09e470e5f24ad2bb8035af516118252 not found: ID does not exist" containerID="aad578a8ba1d909be721ccd1999fd710e09e470e5f24ad2bb8035af516118252"
Oct 31 22:18:42.823450 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.823351    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aad578a8ba1d909be721ccd1999fd710e09e470e5f24ad2bb8035af516118252"} err="failed to get container status \"aad578a8ba1d909be721ccd1999fd710e09e470e5f24ad2bb8035af516118252\": rpc error: code = NotFound desc = could not find container \"aad578a8ba1d909be721ccd1999fd710e09e470e5f24ad2bb8035af516118252\": container with ID starting with aad578a8ba1d909be721ccd1999fd710e09e470e5f24ad2bb8035af516118252 not found: ID does not exist"
Oct 31 22:18:42.823450 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.823377    4460 scope.go:122] "RemoveContainer" containerID="6abf9618c0148ceb6158fe31ae2e799e97136ead6f17693dd0fb2de1ca58fa70"
Oct 31 22:18:42.825473 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.825442    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-1b0cce-7f88b8d6dc-v4qx7"]
Oct 31 22:18:42.833692 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.833673    4460 scope.go:122] "RemoveContainer" containerID="6abf9618c0148ceb6158fe31ae2e799e97136ead6f17693dd0fb2de1ca58fa70"
Oct 31 22:18:42.834015 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:18:42.833991    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6abf9618c0148ceb6158fe31ae2e799e97136ead6f17693dd0fb2de1ca58fa70\": container with ID starting with 6abf9618c0148ceb6158fe31ae2e799e97136ead6f17693dd0fb2de1ca58fa70 not found: ID does not exist" containerID="6abf9618c0148ceb6158fe31ae2e799e97136ead6f17693dd0fb2de1ca58fa70"
Oct 31 22:18:42.834101 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.834024    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6abf9618c0148ceb6158fe31ae2e799e97136ead6f17693dd0fb2de1ca58fa70"} err="failed to get container status \"6abf9618c0148ceb6158fe31ae2e799e97136ead6f17693dd0fb2de1ca58fa70\": rpc error: code = NotFound desc = could not find container \"6abf9618c0148ceb6158fe31ae2e799e97136ead6f17693dd0fb2de1ca58fa70\": container with ID starting with 6abf9618c0148ceb6158fe31ae2e799e97136ead6f17693dd0fb2de1ca58fa70 not found: ID does not exist"
Oct 31 22:18:42.924605 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.924505    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-696f9a-fc885d6c9-pnrxl"]
Oct 31 22:18:42.929655 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.929622    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-696f9a-fc885d6c9-pnrxl"]
Oct 31 22:18:42.932442 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.932415    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:42.967842 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.967788    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-696f9a-67f544df69-gb7nm"]
Oct 31 22:18:42.973742 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:42.973714    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-696f9a-67f544df69-gb7nm"]
Oct 31 22:18:43.052780 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:43.052698    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:43.218913 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:43.218832    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:44.225269 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:44.225233    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="869e1829-ba34-4ed7-bd97-37aa003a0d0f" path="/var/lib/kubelet/pods/869e1829-ba34-4ed7-bd97-37aa003a0d0f/volumes"
Oct 31 22:18:44.225806 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:44.225789    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="de079877-5ceb-4370-b8f6-9f6644de83b3" path="/var/lib/kubelet/pods/de079877-5ceb-4370-b8f6-9f6644de83b3/volumes"
Oct 31 22:18:45.030220 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:45.030186    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:46.837686 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:46.837639    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:47.789997 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:47.789884    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-1b0cce-7996cb486f-vpwrv" podUID="47f0f7a9-1ed4-4046-898f-29220ce6f46b" containerName="payload-processing" containerID="cri-o://f4004f5c1898b425d9781d619d9cbc1fb2b500e596cc18fbbedc4ba10df14f7d" gracePeriod=25
Oct 31 22:18:47.826544 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:47.826500    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1b0cce-7f88b8d6dc-v4qx7" podUID="aa2bcc59-339d-499c-84f0-b2a6eb365ec0" containerName="payload-pre-processing" containerID="cri-o://c511e565f8e548d6d04167b629cf48e38975dadbe9ac5cab4c4c058584fee7d5" gracePeriod=25
Oct 31 22:18:48.059555 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:48.059494    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-1b0cce-7996cb486f-vpwrv"
Oct 31 22:18:48.138810 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:48.138771    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/47f0f7a9-1ed4-4046-898f-29220ce6f46b-kube-api-access-2nx5f\" (UniqueName: \"kubernetes.io/projected/47f0f7a9-1ed4-4046-898f-29220ce6f46b-kube-api-access-2nx5f\") pod \"47f0f7a9-1ed4-4046-898f-29220ce6f46b\" (UID: \"47f0f7a9-1ed4-4046-898f-29220ce6f46b\") "
Oct 31 22:18:48.139006 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:48.138840    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/47f0f7a9-1ed4-4046-898f-29220ce6f46b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47f0f7a9-1ed4-4046-898f-29220ce6f46b-plugins-config-volume\") pod \"47f0f7a9-1ed4-4046-898f-29220ce6f46b\" (UID: \"47f0f7a9-1ed4-4046-898f-29220ce6f46b\") "
Oct 31 22:18:48.139257 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:48.139231    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47f0f7a9-1ed4-4046-898f-29220ce6f46b-plugins-config-volume" pod "47f0f7a9-1ed4-4046-898f-29220ce6f46b" (UID: "47f0f7a9-1ed4-4046-898f-29220ce6f46b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:18:48.139519 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:48.139487    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47f0f7a9-1ed4-4046-898f-29220ce6f46b-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:18:48.140982 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:48.140955    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47f0f7a9-1ed4-4046-898f-29220ce6f46b-kube-api-access-2nx5f" pod "47f0f7a9-1ed4-4046-898f-29220ce6f46b" (UID: "47f0f7a9-1ed4-4046-898f-29220ce6f46b"). InnerVolumeSpecName "kube-api-access-2nx5f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:18:48.240685 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:48.240647    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nx5f\" (UniqueName: \"kubernetes.io/projected/47f0f7a9-1ed4-4046-898f-29220ce6f46b-kube-api-access-2nx5f\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:18:48.579638 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:48.579597    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:48.855132 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:48.855050    4460 generic.go:356] "Generic (PLEG): container finished" podID="47f0f7a9-1ed4-4046-898f-29220ce6f46b" containerID="f4004f5c1898b425d9781d619d9cbc1fb2b500e596cc18fbbedc4ba10df14f7d" exitCode=0
Oct 31 22:18:48.855132 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:48.855101    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-1b0cce-7996cb486f-vpwrv"
Oct 31 22:18:48.855132 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:48.855108    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-1b0cce-7996cb486f-vpwrv" event={"ID":"47f0f7a9-1ed4-4046-898f-29220ce6f46b","Type":"ContainerDied","Data":"f4004f5c1898b425d9781d619d9cbc1fb2b500e596cc18fbbedc4ba10df14f7d"}
Oct 31 22:18:48.855402 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:48.855157    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-1b0cce-7996cb486f-vpwrv" event={"ID":"47f0f7a9-1ed4-4046-898f-29220ce6f46b","Type":"ContainerDied","Data":"9718e0e63f3e0d116c66bf30f8580dab0d7e401e677a8c5bd66555f24b1bf85c"}
Oct 31 22:18:48.855402 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:48.855181    4460 scope.go:122] "RemoveContainer" containerID="f4004f5c1898b425d9781d619d9cbc1fb2b500e596cc18fbbedc4ba10df14f7d"
Oct 31 22:18:48.865230 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:48.865210    4460 scope.go:122] "RemoveContainer" containerID="f4004f5c1898b425d9781d619d9cbc1fb2b500e596cc18fbbedc4ba10df14f7d"
Oct 31 22:18:48.865512 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:18:48.865494    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4004f5c1898b425d9781d619d9cbc1fb2b500e596cc18fbbedc4ba10df14f7d\": container with ID starting with f4004f5c1898b425d9781d619d9cbc1fb2b500e596cc18fbbedc4ba10df14f7d not found: ID does not exist" containerID="f4004f5c1898b425d9781d619d9cbc1fb2b500e596cc18fbbedc4ba10df14f7d"
Oct 31 22:18:48.865568 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:48.865520    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4004f5c1898b425d9781d619d9cbc1fb2b500e596cc18fbbedc4ba10df14f7d"} err="failed to get container status \"f4004f5c1898b425d9781d619d9cbc1fb2b500e596cc18fbbedc4ba10df14f7d\": rpc error: code = NotFound desc = could not find container \"f4004f5c1898b425d9781d619d9cbc1fb2b500e596cc18fbbedc4ba10df14f7d\": container with ID starting with f4004f5c1898b425d9781d619d9cbc1fb2b500e596cc18fbbedc4ba10df14f7d not found: ID does not exist"
Oct 31 22:18:48.912096 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:48.912065    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-1b0cce-7996cb486f-vpwrv"]
Oct 31 22:18:48.925603 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:48.925572    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-1b0cce-7996cb486f-vpwrv"]
Oct 31 22:18:50.084980 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:50.084906    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:50.227176 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:50.227139    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="47f0f7a9-1ed4-4046-898f-29220ce6f46b" path="/var/lib/kubelet/pods/47f0f7a9-1ed4-4046-898f-29220ce6f46b/volumes"
Oct 31 22:18:50.357789 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:50.357709    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-4a73b1-77b884cb9-g55qb"]
Oct 31 22:18:50.391151 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:50.391117    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-4a73b1-7f9ccbd4c9-j9ws8"]
Oct 31 22:18:50.499757 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:50.499719    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:50.623094 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:50.623006    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:50.730483 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:50.730440    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:51.203380 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:51.203342    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:51.410343 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:51.410307    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:51.455198 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:18:51.455117    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:18:52.540503 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:52.540466    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:54.214057 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:54.214014    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:55.019538 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:55.019498    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1b0cce-7f88b8d6dc-v4qx7" podUID="aa2bcc59-339d-499c-84f0-b2a6eb365ec0" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:18:55.358786 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:55.358689    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-b-4a73b1-77b884cb9-g55qb" podUID="48f52b01-f63f-47c2-b0a5-d07370934c67" containerName="payload-processing" containerID="cri-o://5c773c0198e8baf9f670550ca3cab563e260118751e84d5c89d52869889286c2" gracePeriod=25
Oct 31 22:18:55.392450 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:55.392406    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4a73b1-7f9ccbd4c9-j9ws8" podUID="e438fbbd-062a-4ba7-9c8e-fe954d0cf55b" containerName="payload-pre-processing" containerID="cri-o://f4a409df07b78cb97a4b37f78725d2cb5ad2e6fdb0ce2c7c8064c99c96842a81" gracePeriod=25
Oct 31 22:18:55.531980 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:55.531912    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:55.877100 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:55.877062    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:55.990114 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:55.990080    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:56.102577 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:56.102527    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:56.536764 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:56.536730    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:56.646694 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:56.646658    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:56.753888 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:56.753850    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:57.152824 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:57.152789    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:57.868927 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:57.868893    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:57.983848 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:57.983806    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:58.161658 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:58.161580    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"]
Oct 31 22:18:58.161861 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:58.161837    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6" podUID="ae15a792-70d4-40c4-8222-5a81b6e885be" containerName="istio-proxy" containerID="cri-o://409dccab454a42ff858e8d77db1b0726d923c9de0d0f8239eac5d7201c917ad3" gracePeriod=30
Oct 31 22:18:58.190464 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:58.190426    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:58.316906 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:58.316869    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:58.641241 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:58.641207    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:18:59.723635 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:18:59.723602    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 22:19:00.008412 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:00.008332    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:01.485209 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:01.485163    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:02.998396 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:02.998361    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:03.435432 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.435404    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 22:19:03.441578 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.441550    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-workload-socket\") pod \"ae15a792-70d4-40c4-8222-5a81b6e885be\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") "
Oct 31 22:19:03.441703 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.441587    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ae15a792-70d4-40c4-8222-5a81b6e885be-istio-token\" (UniqueName: \"kubernetes.io/projected/ae15a792-70d4-40c4-8222-5a81b6e885be-istio-token\") pod \"ae15a792-70d4-40c4-8222-5a81b6e885be\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") "
Oct 31 22:19:03.441703 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.441613    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-workload-certs\") pod \"ae15a792-70d4-40c4-8222-5a81b6e885be\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") "
Oct 31 22:19:03.441703 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.441642    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ae15a792-70d4-40c4-8222-5a81b6e885be-kube-api-access-g7mt6\" (UniqueName: \"kubernetes.io/projected/ae15a792-70d4-40c4-8222-5a81b6e885be-kube-api-access-g7mt6\") pod \"ae15a792-70d4-40c4-8222-5a81b6e885be\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") "
Oct 31 22:19:03.441703 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.441675    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-istio-data\") pod \"ae15a792-70d4-40c4-8222-5a81b6e885be\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") "
Oct 31 22:19:03.441909 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.441754    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-credential-socket\") pod \"ae15a792-70d4-40c4-8222-5a81b6e885be\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") "
Oct 31 22:19:03.441909 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.441823    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-workload-socket" pod "ae15a792-70d4-40c4-8222-5a81b6e885be" (UID: "ae15a792-70d4-40c4-8222-5a81b6e885be"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:19:03.441909 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.441835    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-istio-envoy\") pod \"ae15a792-70d4-40c4-8222-5a81b6e885be\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") "
Oct 31 22:19:03.442110 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.441973    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ae15a792-70d4-40c4-8222-5a81b6e885be-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae15a792-70d4-40c4-8222-5a81b6e885be-istio-podinfo\") pod \"ae15a792-70d4-40c4-8222-5a81b6e885be\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") "
Oct 31 22:19:03.442110 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.442017    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-workload-certs" pod "ae15a792-70d4-40c4-8222-5a81b6e885be" (UID: "ae15a792-70d4-40c4-8222-5a81b6e885be"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:19:03.442110 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.442056    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-istio-data" pod "ae15a792-70d4-40c4-8222-5a81b6e885be" (UID: "ae15a792-70d4-40c4-8222-5a81b6e885be"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:19:03.442110 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.442095    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-credential-socket" pod "ae15a792-70d4-40c4-8222-5a81b6e885be" (UID: "ae15a792-70d4-40c4-8222-5a81b6e885be"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:19:03.442325 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.442102    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ae15a792-70d4-40c4-8222-5a81b6e885be-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae15a792-70d4-40c4-8222-5a81b6e885be-istiod-ca-cert\") pod \"ae15a792-70d4-40c4-8222-5a81b6e885be\" (UID: \"ae15a792-70d4-40c4-8222-5a81b6e885be\") "
Oct 31 22:19:03.442448 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.442427    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae15a792-70d4-40c4-8222-5a81b6e885be-istiod-ca-cert" pod "ae15a792-70d4-40c4-8222-5a81b6e885be" (UID: "ae15a792-70d4-40c4-8222-5a81b6e885be"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:19:03.442747 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.442721    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae15a792-70d4-40c4-8222-5a81b6e885be-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:03.442747 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.442749    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:03.442994 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.442764    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:03.442994 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.442779    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:03.442994 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.442793    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:03.444018 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.443986    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae15a792-70d4-40c4-8222-5a81b6e885be-istio-token" pod "ae15a792-70d4-40c4-8222-5a81b6e885be" (UID: "ae15a792-70d4-40c4-8222-5a81b6e885be"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:19:03.444134 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.444025    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-istio-envoy" pod "ae15a792-70d4-40c4-8222-5a81b6e885be" (UID: "ae15a792-70d4-40c4-8222-5a81b6e885be"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:19:03.444196 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.444139    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ae15a792-70d4-40c4-8222-5a81b6e885be-istio-podinfo" pod "ae15a792-70d4-40c4-8222-5a81b6e885be" (UID: "ae15a792-70d4-40c4-8222-5a81b6e885be"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 22:19:03.444240 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.444217    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae15a792-70d4-40c4-8222-5a81b6e885be-kube-api-access-g7mt6" pod "ae15a792-70d4-40c4-8222-5a81b6e885be" (UID: "ae15a792-70d4-40c4-8222-5a81b6e885be"). InnerVolumeSpecName "kube-api-access-g7mt6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:19:03.543888 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.543800    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae15a792-70d4-40c4-8222-5a81b6e885be-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:03.543888 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.543835    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7mt6\" (UniqueName: \"kubernetes.io/projected/ae15a792-70d4-40c4-8222-5a81b6e885be-kube-api-access-g7mt6\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:03.543888 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.543845    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae15a792-70d4-40c4-8222-5a81b6e885be-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:03.543888 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.543854    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae15a792-70d4-40c4-8222-5a81b6e885be-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:03.597101 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.597063    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:03.598508 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.598484    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"]
Oct 31 22:19:03.611559 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.611524    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4a73b1-7f9ccbd4c9-j9ws8" podUID="e438fbbd-062a-4ba7-9c8e-fe954d0cf55b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:19:03.611696 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.611583    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-b-4a73b1-77b884cb9-g55qb" podUID="48f52b01-f63f-47c2-b0a5-d07370934c67" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:19:03.613713 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.613690    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"]
Oct 31 22:19:03.723753 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.723716    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:03.833477 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.833384    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:03.950060 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.950023    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:03.953584 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.953557    4460 generic.go:356] "Generic (PLEG): container finished" podID="ae15a792-70d4-40c4-8222-5a81b6e885be" containerID="409dccab454a42ff858e8d77db1b0726d923c9de0d0f8239eac5d7201c917ad3" exitCode=0
Oct 31 22:19:03.953708 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.953606    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"
Oct 31 22:19:03.953708 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.953632    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6" event={"ID":"ae15a792-70d4-40c4-8222-5a81b6e885be","Type":"ContainerDied","Data":"409dccab454a42ff858e8d77db1b0726d923c9de0d0f8239eac5d7201c917ad3"}
Oct 31 22:19:03.953708 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.953684    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6" event={"ID":"ae15a792-70d4-40c4-8222-5a81b6e885be","Type":"ContainerDied","Data":"003ab786806022397c85ebdb77d7a3e6cfccace56f76b27bc31193c7c2fa6a93"}
Oct 31 22:19:03.953858 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.953712    4460 scope.go:122] "RemoveContainer" containerID="409dccab454a42ff858e8d77db1b0726d923c9de0d0f8239eac5d7201c917ad3"
Oct 31 22:19:03.964982 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.964965    4460 scope.go:122] "RemoveContainer" containerID="e5346db7c8cd8b6dc928e9fc6064bcce5d46b3cd488c92e68e7104ecba4b5107"
Oct 31 22:19:03.974246 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.974224    4460 scope.go:122] "RemoveContainer" containerID="409dccab454a42ff858e8d77db1b0726d923c9de0d0f8239eac5d7201c917ad3"
Oct 31 22:19:03.974513 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:19:03.974488    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"409dccab454a42ff858e8d77db1b0726d923c9de0d0f8239eac5d7201c917ad3\": container with ID starting with 409dccab454a42ff858e8d77db1b0726d923c9de0d0f8239eac5d7201c917ad3 not found: ID does not exist" containerID="409dccab454a42ff858e8d77db1b0726d923c9de0d0f8239eac5d7201c917ad3"
Oct 31 22:19:03.974600 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.974561    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"409dccab454a42ff858e8d77db1b0726d923c9de0d0f8239eac5d7201c917ad3"} err="failed to get container status \"409dccab454a42ff858e8d77db1b0726d923c9de0d0f8239eac5d7201c917ad3\": rpc error: code = NotFound desc = could not find container \"409dccab454a42ff858e8d77db1b0726d923c9de0d0f8239eac5d7201c917ad3\": container with ID starting with 409dccab454a42ff858e8d77db1b0726d923c9de0d0f8239eac5d7201c917ad3 not found: ID does not exist"
Oct 31 22:19:03.974600 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.974582    4460 scope.go:122] "RemoveContainer" containerID="e5346db7c8cd8b6dc928e9fc6064bcce5d46b3cd488c92e68e7104ecba4b5107"
Oct 31 22:19:03.974867 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:19:03.974844    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5346db7c8cd8b6dc928e9fc6064bcce5d46b3cd488c92e68e7104ecba4b5107\": container with ID starting with e5346db7c8cd8b6dc928e9fc6064bcce5d46b3cd488c92e68e7104ecba4b5107 not found: ID does not exist" containerID="e5346db7c8cd8b6dc928e9fc6064bcce5d46b3cd488c92e68e7104ecba4b5107"
Oct 31 22:19:03.974970 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:03.974875    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5346db7c8cd8b6dc928e9fc6064bcce5d46b3cd488c92e68e7104ecba4b5107"} err="failed to get container status \"e5346db7c8cd8b6dc928e9fc6064bcce5d46b3cd488c92e68e7104ecba4b5107\": rpc error: code = NotFound desc = could not find container \"e5346db7c8cd8b6dc928e9fc6064bcce5d46b3cd488c92e68e7104ecba4b5107\": container with ID starting with e5346db7c8cd8b6dc928e9fc6064bcce5d46b3cd488c92e68e7104ecba4b5107 not found: ID does not exist"
Oct 31 22:19:04.014131 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:04.014100    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"]
Oct 31 22:19:04.039030 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:04.038988    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-c744c0-openshift-default-5dfdb574cf-6mrl6"]
Oct 31 22:19:04.224862 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:04.224781    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ae15a792-70d4-40c4-8222-5a81b6e885be" path="/var/lib/kubelet/pods/ae15a792-70d4-40c4-8222-5a81b6e885be/volumes"
Oct 31 22:19:04.383750 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:04.383716    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:04.494729 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:04.494643    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:04.614098 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:04.609858    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:04.729446 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:04.729407    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:05.020240 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:05.020202    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1b0cce-7f88b8d6dc-v4qx7" podUID="aa2bcc59-339d-499c-84f0-b2a6eb365ec0" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:19:06.282605 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:06.282565    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:07.954570 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:07.954543    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:07.979934 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:07.979900    4460 generic.go:356] "Generic (PLEG): container finished" podID="aa2bcc59-339d-499c-84f0-b2a6eb365ec0" containerID="c511e565f8e548d6d04167b629cf48e38975dadbe9ac5cab4c4c058584fee7d5" exitCode=0
Oct 31 22:19:07.980102 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:07.979971    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1b0cce-7f88b8d6dc-v4qx7" event={"ID":"aa2bcc59-339d-499c-84f0-b2a6eb365ec0","Type":"ContainerDied","Data":"c511e565f8e548d6d04167b629cf48e38975dadbe9ac5cab4c4c058584fee7d5"}
Oct 31 22:19:08.115274 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:08.115247    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1b0cce-7f88b8d6dc-v4qx7"
Oct 31 22:19:08.205989 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:08.205953    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa2bcc59-339d-499c-84f0-b2a6eb365ec0-kube-api-access-9ksnp\" (UniqueName: \"kubernetes.io/projected/aa2bcc59-339d-499c-84f0-b2a6eb365ec0-kube-api-access-9ksnp\") pod \"aa2bcc59-339d-499c-84f0-b2a6eb365ec0\" (UID: \"aa2bcc59-339d-499c-84f0-b2a6eb365ec0\") "
Oct 31 22:19:08.206185 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:08.206019    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aa2bcc59-339d-499c-84f0-b2a6eb365ec0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa2bcc59-339d-499c-84f0-b2a6eb365ec0-plugins-config-volume\") pod \"aa2bcc59-339d-499c-84f0-b2a6eb365ec0\" (UID: \"aa2bcc59-339d-499c-84f0-b2a6eb365ec0\") "
Oct 31 22:19:08.206405 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:08.206379    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa2bcc59-339d-499c-84f0-b2a6eb365ec0-plugins-config-volume" pod "aa2bcc59-339d-499c-84f0-b2a6eb365ec0" (UID: "aa2bcc59-339d-499c-84f0-b2a6eb365ec0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:19:08.206733 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:08.206709    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa2bcc59-339d-499c-84f0-b2a6eb365ec0-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:08.208045 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:08.208022    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa2bcc59-339d-499c-84f0-b2a6eb365ec0-kube-api-access-9ksnp" pod "aa2bcc59-339d-499c-84f0-b2a6eb365ec0" (UID: "aa2bcc59-339d-499c-84f0-b2a6eb365ec0"). InnerVolumeSpecName "kube-api-access-9ksnp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:19:08.307571 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:08.307534    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ksnp\" (UniqueName: \"kubernetes.io/projected/aa2bcc59-339d-499c-84f0-b2a6eb365ec0-kube-api-access-9ksnp\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:08.987432 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:08.987398    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1b0cce-7f88b8d6dc-v4qx7"
Oct 31 22:19:08.988004 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:08.987439    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-1b0cce-7f88b8d6dc-v4qx7" event={"ID":"aa2bcc59-339d-499c-84f0-b2a6eb365ec0","Type":"ContainerDied","Data":"efbb05c78e787d214483895f299e471cce44961de2900250ebd5d5b1ddf54e21"}
Oct 31 22:19:08.988004 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:08.987508    4460 scope.go:122] "RemoveContainer" containerID="c511e565f8e548d6d04167b629cf48e38975dadbe9ac5cab4c4c058584fee7d5"
Oct 31 22:19:09.044229 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:09.044202    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-1b0cce-7f88b8d6dc-v4qx7"]
Oct 31 22:19:09.057721 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:09.057695    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-1b0cce-7f88b8d6dc-v4qx7"]
Oct 31 22:19:09.397234 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:09.397203    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-cc45a7-86f7d9c8c9-lpr6x"]
Oct 31 22:19:09.489935 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:09.489898    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:09.638958 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:09.622734    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:10.226084 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:10.226053    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aa2bcc59-339d-499c-84f0-b2a6eb365ec0" path="/var/lib/kubelet/pods/aa2bcc59-339d-499c-84f0-b2a6eb365ec0/volumes"
Oct 31 22:19:11.118134 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:11.118093    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:12.609319 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:12.609277    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:13.613165 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:13.613125    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4a73b1-7f9ccbd4c9-j9ws8" podUID="e438fbbd-062a-4ba7-9c8e-fe954d0cf55b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:19:13.613165 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:13.613142    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-b-4a73b1-77b884cb9-g55qb" podUID="48f52b01-f63f-47c2-b0a5-d07370934c67" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:19:14.215522 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:14.215482    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:14.398089 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:14.398050    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cc45a7-86f7d9c8c9-lpr6x" podUID="2b6a669d-e3cb-405b-a1c4-6fdfceb78990" containerName="payload-pre-processing" containerID="cri-o://57633fd1ff45c42392db0018098b0cf779ab0b90dd4c9fb64f88e3a8e6086d78" gracePeriod=25
Oct 31 22:19:14.988320 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:14.988281    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5c54fc-7467846795-qndhq"]
Oct 31 22:19:15.096912 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:15.096861    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:15.205291 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:15.205254    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:15.727011 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:15.726984    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-4a73b1-77b884cb9-g55qb"
Oct 31 22:19:15.771326 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:15.771295    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4a73b1-7f9ccbd4c9-j9ws8"
Oct 31 22:19:15.814220 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:15.814192    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/48f52b01-f63f-47c2-b0a5-d07370934c67-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48f52b01-f63f-47c2-b0a5-d07370934c67-plugins-config-volume\") pod \"48f52b01-f63f-47c2-b0a5-d07370934c67\" (UID: \"48f52b01-f63f-47c2-b0a5-d07370934c67\") "
Oct 31 22:19:15.814370 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:15.814256    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/48f52b01-f63f-47c2-b0a5-d07370934c67-kube-api-access-z7bl4\" (UniqueName: \"kubernetes.io/projected/48f52b01-f63f-47c2-b0a5-d07370934c67-kube-api-access-z7bl4\") pod \"48f52b01-f63f-47c2-b0a5-d07370934c67\" (UID: \"48f52b01-f63f-47c2-b0a5-d07370934c67\") "
Oct 31 22:19:15.814644 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:15.814618    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48f52b01-f63f-47c2-b0a5-d07370934c67-plugins-config-volume" pod "48f52b01-f63f-47c2-b0a5-d07370934c67" (UID: "48f52b01-f63f-47c2-b0a5-d07370934c67"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:19:15.815305 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:15.815277    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48f52b01-f63f-47c2-b0a5-d07370934c67-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:15.817118 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:15.817092    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48f52b01-f63f-47c2-b0a5-d07370934c67-kube-api-access-z7bl4" pod "48f52b01-f63f-47c2-b0a5-d07370934c67" (UID: "48f52b01-f63f-47c2-b0a5-d07370934c67"). InnerVolumeSpecName "kube-api-access-z7bl4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:19:15.821358 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:15.821336    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:15.915725 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:15.915693    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e438fbbd-062a-4ba7-9c8e-fe954d0cf55b-kube-api-access-flzxg\" (UniqueName: \"kubernetes.io/projected/e438fbbd-062a-4ba7-9c8e-fe954d0cf55b-kube-api-access-flzxg\") pod \"e438fbbd-062a-4ba7-9c8e-fe954d0cf55b\" (UID: \"e438fbbd-062a-4ba7-9c8e-fe954d0cf55b\") "
Oct 31 22:19:15.915918 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:15.915903    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e438fbbd-062a-4ba7-9c8e-fe954d0cf55b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e438fbbd-062a-4ba7-9c8e-fe954d0cf55b-plugins-config-volume\") pod \"e438fbbd-062a-4ba7-9c8e-fe954d0cf55b\" (UID: \"e438fbbd-062a-4ba7-9c8e-fe954d0cf55b\") "
Oct 31 22:19:15.916274 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:15.916245    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e438fbbd-062a-4ba7-9c8e-fe954d0cf55b-plugins-config-volume" pod "e438fbbd-062a-4ba7-9c8e-fe954d0cf55b" (UID: "e438fbbd-062a-4ba7-9c8e-fe954d0cf55b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:19:15.916396 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:15.916291    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7bl4\" (UniqueName: \"kubernetes.io/projected/48f52b01-f63f-47c2-b0a5-d07370934c67-kube-api-access-z7bl4\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:15.917731 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:15.917704    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e438fbbd-062a-4ba7-9c8e-fe954d0cf55b-kube-api-access-flzxg" pod "e438fbbd-062a-4ba7-9c8e-fe954d0cf55b" (UID: "e438fbbd-062a-4ba7-9c8e-fe954d0cf55b"). InnerVolumeSpecName "kube-api-access-flzxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:19:15.930870 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:15.930849    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:16.017613 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:16.017578    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e438fbbd-062a-4ba7-9c8e-fe954d0cf55b-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:16.017613 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:16.017609    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-flzxg\" (UniqueName: \"kubernetes.io/projected/e438fbbd-062a-4ba7-9c8e-fe954d0cf55b-kube-api-access-flzxg\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:16.036012 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:16.035977    4460 generic.go:356] "Generic (PLEG): container finished" podID="48f52b01-f63f-47c2-b0a5-d07370934c67" containerID="5c773c0198e8baf9f670550ca3cab563e260118751e84d5c89d52869889286c2" exitCode=0
Oct 31 22:19:16.036012 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:16.036007    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-4a73b1-77b884cb9-g55qb"
Oct 31 22:19:16.036297 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:16.036060    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-4a73b1-77b884cb9-g55qb" event={"ID":"48f52b01-f63f-47c2-b0a5-d07370934c67","Type":"ContainerDied","Data":"5c773c0198e8baf9f670550ca3cab563e260118751e84d5c89d52869889286c2"}
Oct 31 22:19:16.036297 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:16.036130    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-4a73b1-77b884cb9-g55qb" event={"ID":"48f52b01-f63f-47c2-b0a5-d07370934c67","Type":"ContainerDied","Data":"817c33831dc1e8105b36023fb07070800e635cd7c8b4a0380bf1e4d9975d327b"}
Oct 31 22:19:16.036297 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:16.036172    4460 scope.go:122] "RemoveContainer" containerID="5c773c0198e8baf9f670550ca3cab563e260118751e84d5c89d52869889286c2"
Oct 31 22:19:16.037571 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:16.037330    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:16.037926 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:16.037902    4460 generic.go:356] "Generic (PLEG): container finished" podID="e438fbbd-062a-4ba7-9c8e-fe954d0cf55b" containerID="f4a409df07b78cb97a4b37f78725d2cb5ad2e6fdb0ce2c7c8064c99c96842a81" exitCode=0
Oct 31 22:19:16.038068 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:16.037954    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4a73b1-7f9ccbd4c9-j9ws8" event={"ID":"e438fbbd-062a-4ba7-9c8e-fe954d0cf55b","Type":"ContainerDied","Data":"f4a409df07b78cb97a4b37f78725d2cb5ad2e6fdb0ce2c7c8064c99c96842a81"}
Oct 31 22:19:16.038068 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:16.037982    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4a73b1-7f9ccbd4c9-j9ws8"
Oct 31 22:19:16.038068 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:16.038009    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-4a73b1-7f9ccbd4c9-j9ws8" event={"ID":"e438fbbd-062a-4ba7-9c8e-fe954d0cf55b","Type":"ContainerDied","Data":"5763aa2ad4b0d4d57c50a9cde3dfae654b97f04996a4da78d07ec648aad50b85"}
Oct 31 22:19:16.048279 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:16.048231    4460 scope.go:122] "RemoveContainer" containerID="5c773c0198e8baf9f670550ca3cab563e260118751e84d5c89d52869889286c2"
Oct 31 22:19:16.048643 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:19:16.048613    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c773c0198e8baf9f670550ca3cab563e260118751e84d5c89d52869889286c2\": container with ID starting with 5c773c0198e8baf9f670550ca3cab563e260118751e84d5c89d52869889286c2 not found: ID does not exist" containerID="5c773c0198e8baf9f670550ca3cab563e260118751e84d5c89d52869889286c2"
Oct 31 22:19:16.048738 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:16.048651    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c773c0198e8baf9f670550ca3cab563e260118751e84d5c89d52869889286c2"} err="failed to get container status \"5c773c0198e8baf9f670550ca3cab563e260118751e84d5c89d52869889286c2\": rpc error: code = NotFound desc = could not find container \"5c773c0198e8baf9f670550ca3cab563e260118751e84d5c89d52869889286c2\": container with ID starting with 5c773c0198e8baf9f670550ca3cab563e260118751e84d5c89d52869889286c2 not found: ID does not exist"
Oct 31 22:19:16.048738 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:16.048669    4460 scope.go:122] "RemoveContainer" containerID="f4a409df07b78cb97a4b37f78725d2cb5ad2e6fdb0ce2c7c8064c99c96842a81"
Oct 31 22:19:16.058311 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:16.058291    4460 scope.go:122] "RemoveContainer" containerID="f4a409df07b78cb97a4b37f78725d2cb5ad2e6fdb0ce2c7c8064c99c96842a81"
Oct 31 22:19:16.058637 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:19:16.058617    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4a409df07b78cb97a4b37f78725d2cb5ad2e6fdb0ce2c7c8064c99c96842a81\": container with ID starting with f4a409df07b78cb97a4b37f78725d2cb5ad2e6fdb0ce2c7c8064c99c96842a81 not found: ID does not exist" containerID="f4a409df07b78cb97a4b37f78725d2cb5ad2e6fdb0ce2c7c8064c99c96842a81"
Oct 31 22:19:16.058707 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:16.058649    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4a409df07b78cb97a4b37f78725d2cb5ad2e6fdb0ce2c7c8064c99c96842a81"} err="failed to get container status \"f4a409df07b78cb97a4b37f78725d2cb5ad2e6fdb0ce2c7c8064c99c96842a81\": rpc error: code = NotFound desc = could not find container \"f4a409df07b78cb97a4b37f78725d2cb5ad2e6fdb0ce2c7c8064c99c96842a81\": container with ID starting with f4a409df07b78cb97a4b37f78725d2cb5ad2e6fdb0ce2c7c8064c99c96842a81 not found: ID does not exist"
Oct 31 22:19:16.093302 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:16.093261    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-4a73b1-7f9ccbd4c9-j9ws8"]
Oct 31 22:19:16.097698 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:16.097666    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-4a73b1-7f9ccbd4c9-j9ws8"]
Oct 31 22:19:16.136397 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:16.136365    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-4a73b1-77b884cb9-g55qb"]
Oct 31 22:19:16.140160 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:16.140133    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-4a73b1-77b884cb9-g55qb"]
Oct 31 22:19:16.225173 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:16.225143    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="48f52b01-f63f-47c2-b0a5-d07370934c67" path="/var/lib/kubelet/pods/48f52b01-f63f-47c2-b0a5-d07370934c67/volumes"
Oct 31 22:19:16.225591 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:16.225577    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e438fbbd-062a-4ba7-9c8e-fe954d0cf55b" path="/var/lib/kubelet/pods/e438fbbd-062a-4ba7-9c8e-fe954d0cf55b/volumes"
Oct 31 22:19:17.154335 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:17.154298    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:17.610130 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:17.610096    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:17.721446 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:17.721410    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:17.828614 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:17.828575    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:18.228611 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.228579    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:18.620960 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.620895    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n" podUID="4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" containerName="main" containerID="cri-o://2240fefaacf7242c77b2590ed5769617ea2f979ba0174268e69c0c6661e5f5c0" gracePeriod=45
Oct 31 22:19:18.635039 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.634370    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main" containerID="cri-o://0779a636e28bb1df293e46a211d54a017014b80a7df74a50996120ea46660b22" gracePeriod=45
Oct 31 22:19:18.679411 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:19:18.679367    4460 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4cf6bb6a_3a46_41f0_9e4f_f0fabbbc82cf.slice/crio-2240fefaacf7242c77b2590ed5769617ea2f979ba0174268e69c0c6661e5f5c0.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb9cd1fa6_02c8_42af_b733_8f9c326a5b73.slice/crio-0779a636e28bb1df293e46a211d54a017014b80a7df74a50996120ea46660b22.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4cf6bb6a_3a46_41f0_9e4f_f0fabbbc82cf.slice/crio-conmon-2240fefaacf7242c77b2590ed5769617ea2f979ba0174268e69c0c6661e5f5c0.scope\": RecentStats: unable to find data in memory cache]"
Oct 31 22:19:18.898899 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.898872    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 22:19:18.925583 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.925553    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 22:19:18.965178 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.965142    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-dshm\") pod \"b9cd1fa6-02c8-42af-b733-8f9c326a5b73\" (UID: \"b9cd1fa6-02c8-42af-b733-8f9c326a5b73\") "
Oct 31 22:19:18.965402 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.965213    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-tmp-dir\") pod \"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf\" (UID: \"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf\") "
Oct 31 22:19:18.965402 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.965256    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-tls-certs\" (UniqueName: \"kubernetes.io/secret/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-tls-certs\") pod \"b9cd1fa6-02c8-42af-b733-8f9c326a5b73\" (UID: \"b9cd1fa6-02c8-42af-b733-8f9c326a5b73\") "
Oct 31 22:19:18.965402 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.965291    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-dshm\") pod \"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf\" (UID: \"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf\") "
Oct 31 22:19:18.965402 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.965340    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-tmp-dir\") pod \"b9cd1fa6-02c8-42af-b733-8f9c326a5b73\" (UID: \"b9cd1fa6-02c8-42af-b733-8f9c326a5b73\") "
Oct 31 22:19:18.965647 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.965416    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-model-cache\") pod \"b9cd1fa6-02c8-42af-b733-8f9c326a5b73\" (UID: \"b9cd1fa6-02c8-42af-b733-8f9c326a5b73\") "
Oct 31 22:19:18.965647 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.965455    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-tls-certs\" (UniqueName: \"kubernetes.io/secret/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-tls-certs\") pod \"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf\" (UID: \"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf\") "
Oct 31 22:19:18.965647 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.965517    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-home\" (UniqueName: \"kubernetes.io/empty-dir/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-home\") pod \"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf\" (UID: \"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf\") "
Oct 31 22:19:18.965647 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.965544    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-home\" (UniqueName: \"kubernetes.io/empty-dir/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-home\") pod \"b9cd1fa6-02c8-42af-b733-8f9c326a5b73\" (UID: \"b9cd1fa6-02c8-42af-b733-8f9c326a5b73\") "
Oct 31 22:19:18.965647 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.965634    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-kube-api-access-v5zp9\" (UniqueName: \"kubernetes.io/projected/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-kube-api-access-v5zp9\") pod \"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf\" (UID: \"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf\") "
Oct 31 22:19:18.966100 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.965679    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-kube-api-access-6t9jx\" (UniqueName: \"kubernetes.io/projected/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-kube-api-access-6t9jx\") pod \"b9cd1fa6-02c8-42af-b733-8f9c326a5b73\" (UID: \"b9cd1fa6-02c8-42af-b733-8f9c326a5b73\") "
Oct 31 22:19:18.966100 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.965783    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-model-cache\") pod \"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf\" (UID: \"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf\") "
Oct 31 22:19:18.966100 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.965820    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-model-cache" pod "b9cd1fa6-02c8-42af-b733-8f9c326a5b73" (UID: "b9cd1fa6-02c8-42af-b733-8f9c326a5b73"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:19:18.966661 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.966330    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-home" pod "b9cd1fa6-02c8-42af-b733-8f9c326a5b73" (UID: "b9cd1fa6-02c8-42af-b733-8f9c326a5b73"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:19:18.966661 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.966452    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-tmp-dir" pod "b9cd1fa6-02c8-42af-b733-8f9c326a5b73" (UID: "b9cd1fa6-02c8-42af-b733-8f9c326a5b73"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:19:18.966661 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.966463    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-home" pod "4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" (UID: "4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:19:18.967289 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.967260    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-model-cache" pod "4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" (UID: "4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:19:18.967410 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.967374    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-tmp-dir" pod "4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" (UID: "4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:19:18.968246 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.968216    4460 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-model-cache\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:18.968370 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.968250    4460 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-tmp-dir\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:18.968370 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.968265    4460 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-tmp-dir\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:18.968370 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.968278    4460 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-model-cache\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:18.968370 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.968291    4460 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-home\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:18.968370 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.968304    4460 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-home\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:18.971333 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.971307    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-dshm" pod "b9cd1fa6-02c8-42af-b733-8f9c326a5b73" (UID: "b9cd1fa6-02c8-42af-b733-8f9c326a5b73"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:19:18.973494 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.973465    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-kube-api-access-6t9jx" pod "b9cd1fa6-02c8-42af-b733-8f9c326a5b73" (UID: "b9cd1fa6-02c8-42af-b733-8f9c326a5b73"). InnerVolumeSpecName "kube-api-access-6t9jx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:19:18.973628 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.973512    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-kube-api-access-v5zp9" pod "4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" (UID: "4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf"). InnerVolumeSpecName "kube-api-access-v5zp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:19:18.973628 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.973512    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-tls-certs" pod "4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" (UID: "4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 22:19:18.973759 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.973720    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-tls-certs" pod "b9cd1fa6-02c8-42af-b733-8f9c326a5b73" (UID: "b9cd1fa6-02c8-42af-b733-8f9c326a5b73"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 22:19:18.976231 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:18.976208    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-dshm" pod "4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" (UID: "4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:19:19.059755 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:19.059718    4460 generic.go:356] "Generic (PLEG): container finished" podID="4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" containerID="2240fefaacf7242c77b2590ed5769617ea2f979ba0174268e69c0c6661e5f5c0" exitCode=0
Oct 31 22:19:19.059952 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:19.059760    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"
Oct 31 22:19:19.059952 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:19.059804    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n" event={"ID":"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf","Type":"ContainerDied","Data":"2240fefaacf7242c77b2590ed5769617ea2f979ba0174268e69c0c6661e5f5c0"}
Oct 31 22:19:19.059952 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:19.059850    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n" event={"ID":"4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf","Type":"ContainerDied","Data":"0856fa69738f548a68a5add24df6d887d2936f87523d84dd53a8f3d8ca944e27"}
Oct 31 22:19:19.059952 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:19.059872    4460 scope.go:122] "RemoveContainer" containerID="2240fefaacf7242c77b2590ed5769617ea2f979ba0174268e69c0c6661e5f5c0"
Oct 31 22:19:19.061565 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:19.061540    4460 generic.go:356] "Generic (PLEG): container finished" podID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerID="0779a636e28bb1df293e46a211d54a017014b80a7df74a50996120ea46660b22" exitCode=0
Oct 31 22:19:19.061699 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:19.061590    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"
Oct 31 22:19:19.061699 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:19.061626    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" event={"ID":"b9cd1fa6-02c8-42af-b733-8f9c326a5b73","Type":"ContainerDied","Data":"0779a636e28bb1df293e46a211d54a017014b80a7df74a50996120ea46660b22"}
Oct 31 22:19:19.061699 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:19.061673    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt" event={"ID":"b9cd1fa6-02c8-42af-b733-8f9c326a5b73","Type":"ContainerDied","Data":"7d5ef6f2dcaa5dc3dc0d02c96fe4eb0c54de4ec3c950a966ab9a0ef26f7f5cd6"}
Oct 31 22:19:19.069171 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:19.069000    4460 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-dshm\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:19.069171 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:19.069020    4460 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-tls-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:19.069171 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:19.069034    4460 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-dshm\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:19.069171 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:19.069046    4460 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-tls-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:19.069171 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:19.069059    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5zp9\" (UniqueName: \"kubernetes.io/projected/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf-kube-api-access-v5zp9\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:19.069171 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:19.069074    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6t9jx\" (UniqueName: \"kubernetes.io/projected/b9cd1fa6-02c8-42af-b733-8f9c326a5b73-kube-api-access-6t9jx\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:19.070647 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:19.070630    4460 scope.go:122] "RemoveContainer" containerID="2240fefaacf7242c77b2590ed5769617ea2f979ba0174268e69c0c6661e5f5c0"
Oct 31 22:19:19.070994 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:19:19.070954    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2240fefaacf7242c77b2590ed5769617ea2f979ba0174268e69c0c6661e5f5c0\": container with ID starting with 2240fefaacf7242c77b2590ed5769617ea2f979ba0174268e69c0c6661e5f5c0 not found: ID does not exist" containerID="2240fefaacf7242c77b2590ed5769617ea2f979ba0174268e69c0c6661e5f5c0"
Oct 31 22:19:19.071069 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:19.070998    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2240fefaacf7242c77b2590ed5769617ea2f979ba0174268e69c0c6661e5f5c0"} err="failed to get container status \"2240fefaacf7242c77b2590ed5769617ea2f979ba0174268e69c0c6661e5f5c0\": rpc error: code = NotFound desc = could not find container \"2240fefaacf7242c77b2590ed5769617ea2f979ba0174268e69c0c6661e5f5c0\": container with ID starting with 2240fefaacf7242c77b2590ed5769617ea2f979ba0174268e69c0c6661e5f5c0 not found: ID does not exist"
Oct 31 22:19:19.071069 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:19.071024    4460 scope.go:122] "RemoveContainer" containerID="0779a636e28bb1df293e46a211d54a017014b80a7df74a50996120ea46660b22"
Oct 31 22:19:19.079510 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:19.079491    4460 scope.go:122] "RemoveContainer" containerID="0779a636e28bb1df293e46a211d54a017014b80a7df74a50996120ea46660b22"
Oct 31 22:19:19.079759 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:19:19.079741    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0779a636e28bb1df293e46a211d54a017014b80a7df74a50996120ea46660b22\": container with ID starting with 0779a636e28bb1df293e46a211d54a017014b80a7df74a50996120ea46660b22 not found: ID does not exist" containerID="0779a636e28bb1df293e46a211d54a017014b80a7df74a50996120ea46660b22"
Oct 31 22:19:19.079843 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:19.079765    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0779a636e28bb1df293e46a211d54a017014b80a7df74a50996120ea46660b22"} err="failed to get container status \"0779a636e28bb1df293e46a211d54a017014b80a7df74a50996120ea46660b22\": rpc error: code = NotFound desc = could not find container \"0779a636e28bb1df293e46a211d54a017014b80a7df74a50996120ea46660b22\": container with ID starting with 0779a636e28bb1df293e46a211d54a017014b80a7df74a50996120ea46660b22 not found: ID does not exist"
Oct 31 22:19:19.116178 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:19.116145    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"]
Oct 31 22:19:19.121278 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:19.121228    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-cc45a7/premium-w4-cc45a7-kserve-7bb4b8c55c-z8mlt"]
Oct 31 22:19:19.164499 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:19.164433    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"]
Oct 31 22:19:19.168512 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:19.168488    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-cc45a7/distinct-2-w4-cc45a7-kserve-74b7748cb7-6cg8n"]
Oct 31 22:19:19.605322 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:19.605285    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:19.989603 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:19.989503    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5c54fc-7467846795-qndhq" podUID="b7d48a8c-34fc-4974-8cc6-1346a27fcc65" containerName="payload-pre-processing" containerID="cri-o://676db78d3a27c5c5a346f580ff3bce56d0e4bf6e3acd048cbfd30ac114d09951" gracePeriod=25
Oct 31 22:19:20.226374 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:20.226339    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" path="/var/lib/kubelet/pods/4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf/volumes"
Oct 31 22:19:20.226810 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:20.226795    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" path="/var/lib/kubelet/pods/b9cd1fa6-02c8-42af-b733-8f9c326a5b73/volumes"
Oct 31 22:19:21.169342 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:21.169303    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:21.452452 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:19:21.452362    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:19:22.482188 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:22.482147    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:22.984013 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:22.983977    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:23.090901 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:23.090855    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:23.277714 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:23.277683    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:23.374074 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:23.374041    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 22:19:23.587894 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:23.587804    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:24.354021 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:24.353974    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cc45a7-86f7d9c8c9-lpr6x" podUID="2b6a669d-e3cb-405b-a1c4-6fdfceb78990" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:19:24.552933 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:24.552893    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc"]
Oct 31 22:19:24.553217 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:24.553183    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc" podUID="8b369e84-9944-4d3b-9bc1-9aa885694d11" containerName="maas-api" containerID="cri-o://d68596f112e9b44ed304cba2d00e33f292dbf9ae8db65672ece3675581e2f074" gracePeriod=30
Oct 31 22:19:24.621651 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:24.621561    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:24.838965 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:24.825036    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc"
Oct 31 22:19:24.843979 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:24.842234    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8b369e84-9944-4d3b-9bc1-9aa885694d11-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b369e84-9944-4d3b-9bc1-9aa885694d11-maas-api-tls\") pod \"8b369e84-9944-4d3b-9bc1-9aa885694d11\" (UID: \"8b369e84-9944-4d3b-9bc1-9aa885694d11\") "
Oct 31 22:19:24.843979 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:24.842335    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b369e84-9944-4d3b-9bc1-9aa885694d11-kube-api-access-8mf7f\" (UniqueName: \"kubernetes.io/projected/8b369e84-9944-4d3b-9bc1-9aa885694d11-kube-api-access-8mf7f\") pod \"8b369e84-9944-4d3b-9bc1-9aa885694d11\" (UID: \"8b369e84-9944-4d3b-9bc1-9aa885694d11\") "
Oct 31 22:19:24.843979 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:24.842408    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8b369e84-9944-4d3b-9bc1-9aa885694d11-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b369e84-9944-4d3b-9bc1-9aa885694d11-metrics-certs\") pod \"8b369e84-9944-4d3b-9bc1-9aa885694d11\" (UID: \"8b369e84-9944-4d3b-9bc1-9aa885694d11\") "
Oct 31 22:19:24.843979 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:24.842582    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8b369e84-9944-4d3b-9bc1-9aa885694d11-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b369e84-9944-4d3b-9bc1-9aa885694d11-openshift-service-ca\") pod \"8b369e84-9944-4d3b-9bc1-9aa885694d11\" (UID: \"8b369e84-9944-4d3b-9bc1-9aa885694d11\") "
Oct 31 22:19:24.844445 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:24.844128    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b369e84-9944-4d3b-9bc1-9aa885694d11-openshift-service-ca" pod "8b369e84-9944-4d3b-9bc1-9aa885694d11" (UID: "8b369e84-9944-4d3b-9bc1-9aa885694d11"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:19:24.852189 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:24.852150    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b369e84-9944-4d3b-9bc1-9aa885694d11-maas-api-tls" pod "8b369e84-9944-4d3b-9bc1-9aa885694d11" (UID: "8b369e84-9944-4d3b-9bc1-9aa885694d11"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 22:19:24.852709 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:24.852679    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b369e84-9944-4d3b-9bc1-9aa885694d11-metrics-certs" pod "8b369e84-9944-4d3b-9bc1-9aa885694d11" (UID: "8b369e84-9944-4d3b-9bc1-9aa885694d11"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 22:19:24.860037 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:24.856131    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b369e84-9944-4d3b-9bc1-9aa885694d11-kube-api-access-8mf7f" pod "8b369e84-9944-4d3b-9bc1-9aa885694d11" (UID: "8b369e84-9944-4d3b-9bc1-9aa885694d11"). InnerVolumeSpecName "kube-api-access-8mf7f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:19:24.944495 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:24.944405    4460 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b369e84-9944-4d3b-9bc1-9aa885694d11-openshift-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:24.944495 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:24.944439    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b369e84-9944-4d3b-9bc1-9aa885694d11-maas-api-tls\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:24.944495 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:24.944449    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mf7f\" (UniqueName: \"kubernetes.io/projected/8b369e84-9944-4d3b-9bc1-9aa885694d11-kube-api-access-8mf7f\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:24.944495 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:24.944459    4460 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b369e84-9944-4d3b-9bc1-9aa885694d11-metrics-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:25.109383 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:25.109350    4460 generic.go:356] "Generic (PLEG): container finished" podID="8b369e84-9944-4d3b-9bc1-9aa885694d11" containerID="d68596f112e9b44ed304cba2d00e33f292dbf9ae8db65672ece3675581e2f074" exitCode=0
Oct 31 22:19:25.109569 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:25.109406    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc"
Oct 31 22:19:25.109569 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:25.109415    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc" event={"ID":"8b369e84-9944-4d3b-9bc1-9aa885694d11","Type":"ContainerDied","Data":"d68596f112e9b44ed304cba2d00e33f292dbf9ae8db65672ece3675581e2f074"}
Oct 31 22:19:25.109569 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:25.109528    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc" event={"ID":"8b369e84-9944-4d3b-9bc1-9aa885694d11","Type":"ContainerDied","Data":"98bdeb2c7c523f9e305ebc83315ea30c126de9939102879f77242f76c59f844d"}
Oct 31 22:19:25.109569 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:25.109549    4460 scope.go:122] "RemoveContainer" containerID="d68596f112e9b44ed304cba2d00e33f292dbf9ae8db65672ece3675581e2f074"
Oct 31 22:19:25.120360 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:25.120339    4460 scope.go:122] "RemoveContainer" containerID="d68596f112e9b44ed304cba2d00e33f292dbf9ae8db65672ece3675581e2f074"
Oct 31 22:19:25.120669 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:19:25.120646    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d68596f112e9b44ed304cba2d00e33f292dbf9ae8db65672ece3675581e2f074\": container with ID starting with d68596f112e9b44ed304cba2d00e33f292dbf9ae8db65672ece3675581e2f074 not found: ID does not exist" containerID="d68596f112e9b44ed304cba2d00e33f292dbf9ae8db65672ece3675581e2f074"
Oct 31 22:19:25.120717 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:25.120677    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d68596f112e9b44ed304cba2d00e33f292dbf9ae8db65672ece3675581e2f074"} err="failed to get container status \"d68596f112e9b44ed304cba2d00e33f292dbf9ae8db65672ece3675581e2f074\": rpc error: code = NotFound desc = could not find container \"d68596f112e9b44ed304cba2d00e33f292dbf9ae8db65672ece3675581e2f074\": container with ID starting with d68596f112e9b44ed304cba2d00e33f292dbf9ae8db65672ece3675581e2f074 not found: ID does not exist"
Oct 31 22:19:25.165914 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:25.165877    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc"]
Oct 31 22:19:25.186779 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:25.186746    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-cc45a7-7ccfbdff65-vmdnc"]
Oct 31 22:19:25.215145 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:25.215067    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:25.336861 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:25.336820    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:25.471363 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:25.471276    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:25.809052 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:25.809011    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:26.224578 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:26.224499    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8b369e84-9944-4d3b-9bc1-9aa885694d11" path="/var/lib/kubelet/pods/8b369e84-9944-4d3b-9bc1-9aa885694d11/volumes"
Oct 31 22:19:26.933521 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:26.933474    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5c54fc-7467846795-qndhq" podUID="b7d48a8c-34fc-4974-8cc6-1346a27fcc65" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:19:26.953743 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:26.953703    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:27.084545 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:27.084506    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:27.199555 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:27.199451    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:27.313169 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:27.313130    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:28.840514 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:28.840470    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:30.129643 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:30.129607    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:30.238449 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:30.238411    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:30.429205 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:30.429118    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"]
Oct 31 22:19:30.429398 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:30.429369    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv" podUID="390c66e4-91b0-4110-92c3-fd754193823d" containerName="istio-proxy" containerID="cri-o://4e95ab6cd2ec240baa1145ea1a369aee448ed29a08a3733be044b148403b9485" gracePeriod=30
Oct 31 22:19:30.469232 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:30.469192    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:30.578983 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:30.578931    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:32.215786 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:32.215756    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:32.409579 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:32.409536    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:32.520120 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:32.520092    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:32.711302 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:32.711266    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"]
Oct 31 22:19:32.711537 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:32.711506    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk" podUID="3f60d565-50f1-4b9d-959a-014be6e49128" containerName="istio-proxy" containerID="cri-o://b3d79e8dcda1d1ffba9095e19db1120efec82491a5488ddb2692667aedcb0ade" gracePeriod=30
Oct 31 22:19:32.730481 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:32.730438    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:33.899852 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:33.899815    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:34.354217 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:34.354175    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cc45a7-86f7d9c8c9-lpr6x" podUID="2b6a669d-e3cb-405b-a1c4-6fdfceb78990" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:19:34.657636 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:34.657605    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cc45a7-86f7d9c8c9-lpr6x"
Oct 31 22:19:34.786614 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:34.786566    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2b6a669d-e3cb-405b-a1c4-6fdfceb78990-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b6a669d-e3cb-405b-a1c4-6fdfceb78990-plugins-config-volume\") pod \"2b6a669d-e3cb-405b-a1c4-6fdfceb78990\" (UID: \"2b6a669d-e3cb-405b-a1c4-6fdfceb78990\") "
Oct 31 22:19:34.786805 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:34.786756    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b6a669d-e3cb-405b-a1c4-6fdfceb78990-kube-api-access-mvxz7\" (UniqueName: \"kubernetes.io/projected/2b6a669d-e3cb-405b-a1c4-6fdfceb78990-kube-api-access-mvxz7\") pod \"2b6a669d-e3cb-405b-a1c4-6fdfceb78990\" (UID: \"2b6a669d-e3cb-405b-a1c4-6fdfceb78990\") "
Oct 31 22:19:34.786980 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:34.786930    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b6a669d-e3cb-405b-a1c4-6fdfceb78990-plugins-config-volume" pod "2b6a669d-e3cb-405b-a1c4-6fdfceb78990" (UID: "2b6a669d-e3cb-405b-a1c4-6fdfceb78990"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:19:34.787153 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:34.787139    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b6a669d-e3cb-405b-a1c4-6fdfceb78990-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:34.789019 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:34.788993    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b6a669d-e3cb-405b-a1c4-6fdfceb78990-kube-api-access-mvxz7" pod "2b6a669d-e3cb-405b-a1c4-6fdfceb78990" (UID: "2b6a669d-e3cb-405b-a1c4-6fdfceb78990"). InnerVolumeSpecName "kube-api-access-mvxz7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:19:34.887826 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:34.887748    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvxz7\" (UniqueName: \"kubernetes.io/projected/2b6a669d-e3cb-405b-a1c4-6fdfceb78990-kube-api-access-mvxz7\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:35.169808 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.169714    4460 generic.go:356] "Generic (PLEG): container finished" podID="2b6a669d-e3cb-405b-a1c4-6fdfceb78990" containerID="57633fd1ff45c42392db0018098b0cf779ab0b90dd4c9fb64f88e3a8e6086d78" exitCode=0
Oct 31 22:19:35.169808 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.169761    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cc45a7-86f7d9c8c9-lpr6x"
Oct 31 22:19:35.170303 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.169807    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cc45a7-86f7d9c8c9-lpr6x" event={"ID":"2b6a669d-e3cb-405b-a1c4-6fdfceb78990","Type":"ContainerDied","Data":"57633fd1ff45c42392db0018098b0cf779ab0b90dd4c9fb64f88e3a8e6086d78"}
Oct 31 22:19:35.170303 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.169859    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-cc45a7-86f7d9c8c9-lpr6x" event={"ID":"2b6a669d-e3cb-405b-a1c4-6fdfceb78990","Type":"ContainerDied","Data":"3e093649f4f3e93b624f8d8e555218ac1d60cdefbc4b3334eed89aabfd368a06"}
Oct 31 22:19:35.170303 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.169885    4460 scope.go:122] "RemoveContainer" containerID="57633fd1ff45c42392db0018098b0cf779ab0b90dd4c9fb64f88e3a8e6086d78"
Oct 31 22:19:35.179820 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.179799    4460 scope.go:122] "RemoveContainer" containerID="57633fd1ff45c42392db0018098b0cf779ab0b90dd4c9fb64f88e3a8e6086d78"
Oct 31 22:19:35.180133 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:19:35.180110    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57633fd1ff45c42392db0018098b0cf779ab0b90dd4c9fb64f88e3a8e6086d78\": container with ID starting with 57633fd1ff45c42392db0018098b0cf779ab0b90dd4c9fb64f88e3a8e6086d78 not found: ID does not exist" containerID="57633fd1ff45c42392db0018098b0cf779ab0b90dd4c9fb64f88e3a8e6086d78"
Oct 31 22:19:35.180236 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.180139    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57633fd1ff45c42392db0018098b0cf779ab0b90dd4c9fb64f88e3a8e6086d78"} err="failed to get container status \"57633fd1ff45c42392db0018098b0cf779ab0b90dd4c9fb64f88e3a8e6086d78\": rpc error: code = NotFound desc = could not find container \"57633fd1ff45c42392db0018098b0cf779ab0b90dd4c9fb64f88e3a8e6086d78\": container with ID starting with 57633fd1ff45c42392db0018098b0cf779ab0b90dd4c9fb64f88e3a8e6086d78 not found: ID does not exist"
Oct 31 22:19:35.226535 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.226497    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-cc45a7-86f7d9c8c9-lpr6x"]
Oct 31 22:19:35.239981 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.239931    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-cc45a7-86f7d9c8c9-lpr6x"]
Oct 31 22:19:35.540020 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.539987    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:35.700464 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.700399    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:19:35.798042 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.798007    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-workload-certs\") pod \"390c66e4-91b0-4110-92c3-fd754193823d\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") "
Oct 31 22:19:35.798196 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.798064    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/390c66e4-91b0-4110-92c3-fd754193823d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/390c66e4-91b0-4110-92c3-fd754193823d-istiod-ca-cert\") pod \"390c66e4-91b0-4110-92c3-fd754193823d\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") "
Oct 31 22:19:35.798196 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.798170    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-istio-data\") pod \"390c66e4-91b0-4110-92c3-fd754193823d\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") "
Oct 31 22:19:35.798284 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.798202    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/390c66e4-91b0-4110-92c3-fd754193823d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/390c66e4-91b0-4110-92c3-fd754193823d-istio-podinfo\") pod \"390c66e4-91b0-4110-92c3-fd754193823d\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") "
Oct 31 22:19:35.798284 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.798259    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-workload-socket\") pod \"390c66e4-91b0-4110-92c3-fd754193823d\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") "
Oct 31 22:19:35.798284 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.798281    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-istio-envoy\") pod \"390c66e4-91b0-4110-92c3-fd754193823d\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") "
Oct 31 22:19:35.798402 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.798303    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-workload-certs" pod "390c66e4-91b0-4110-92c3-fd754193823d" (UID: "390c66e4-91b0-4110-92c3-fd754193823d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:19:35.798402 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.798312    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-credential-socket\") pod \"390c66e4-91b0-4110-92c3-fd754193823d\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") "
Oct 31 22:19:35.798402 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.798373    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/390c66e4-91b0-4110-92c3-fd754193823d-kube-api-access-jlpjb\" (UniqueName: \"kubernetes.io/projected/390c66e4-91b0-4110-92c3-fd754193823d-kube-api-access-jlpjb\") pod \"390c66e4-91b0-4110-92c3-fd754193823d\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") "
Oct 31 22:19:35.798568 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.798405    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/390c66e4-91b0-4110-92c3-fd754193823d-istio-token\" (UniqueName: \"kubernetes.io/projected/390c66e4-91b0-4110-92c3-fd754193823d-istio-token\") pod \"390c66e4-91b0-4110-92c3-fd754193823d\" (UID: \"390c66e4-91b0-4110-92c3-fd754193823d\") "
Oct 31 22:19:35.798568 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.798418    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-istio-data" pod "390c66e4-91b0-4110-92c3-fd754193823d" (UID: "390c66e4-91b0-4110-92c3-fd754193823d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:19:35.798568 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.798432    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/390c66e4-91b0-4110-92c3-fd754193823d-istiod-ca-cert" pod "390c66e4-91b0-4110-92c3-fd754193823d" (UID: "390c66e4-91b0-4110-92c3-fd754193823d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:19:35.798753 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.798588    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-workload-socket" pod "390c66e4-91b0-4110-92c3-fd754193823d" (UID: "390c66e4-91b0-4110-92c3-fd754193823d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:19:35.798753 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.798725    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-credential-socket" pod "390c66e4-91b0-4110-92c3-fd754193823d" (UID: "390c66e4-91b0-4110-92c3-fd754193823d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:19:35.798968 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.798919    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:35.799037 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.798971    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/390c66e4-91b0-4110-92c3-fd754193823d-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:35.799037 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.798987    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:35.799037 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.799000    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:35.799037 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.799014    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:35.800550 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.800518    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/390c66e4-91b0-4110-92c3-fd754193823d-istio-token" pod "390c66e4-91b0-4110-92c3-fd754193823d" (UID: "390c66e4-91b0-4110-92c3-fd754193823d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:19:35.800670 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.800543    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-istio-envoy" pod "390c66e4-91b0-4110-92c3-fd754193823d" (UID: "390c66e4-91b0-4110-92c3-fd754193823d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:19:35.800670 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.800553    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/390c66e4-91b0-4110-92c3-fd754193823d-istio-podinfo" pod "390c66e4-91b0-4110-92c3-fd754193823d" (UID: "390c66e4-91b0-4110-92c3-fd754193823d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 22:19:35.800670 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.800638    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/390c66e4-91b0-4110-92c3-fd754193823d-kube-api-access-jlpjb" pod "390c66e4-91b0-4110-92c3-fd754193823d" (UID: "390c66e4-91b0-4110-92c3-fd754193823d"). InnerVolumeSpecName "kube-api-access-jlpjb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:19:35.900573 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.900539    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/390c66e4-91b0-4110-92c3-fd754193823d-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:35.900573 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.900568    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlpjb\" (UniqueName: \"kubernetes.io/projected/390c66e4-91b0-4110-92c3-fd754193823d-kube-api-access-jlpjb\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:35.900573 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.900578    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/390c66e4-91b0-4110-92c3-fd754193823d-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:35.900814 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:35.900588    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/390c66e4-91b0-4110-92c3-fd754193823d-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:36.178060 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:36.178020    4460 generic.go:356] "Generic (PLEG): container finished" podID="390c66e4-91b0-4110-92c3-fd754193823d" containerID="4e95ab6cd2ec240baa1145ea1a369aee448ed29a08a3733be044b148403b9485" exitCode=0
Oct 31 22:19:36.178546 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:36.178073    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"
Oct 31 22:19:36.178546 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:36.178102    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv" event={"ID":"390c66e4-91b0-4110-92c3-fd754193823d","Type":"ContainerDied","Data":"4e95ab6cd2ec240baa1145ea1a369aee448ed29a08a3733be044b148403b9485"}
Oct 31 22:19:36.178546 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:36.178154    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv" event={"ID":"390c66e4-91b0-4110-92c3-fd754193823d","Type":"ContainerDied","Data":"716129c29fb41f41ac23fee03580ffc51042037bca7b9629595eff7ce7f743b9"}
Oct 31 22:19:36.178546 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:36.178178    4460 scope.go:122] "RemoveContainer" containerID="4e95ab6cd2ec240baa1145ea1a369aee448ed29a08a3733be044b148403b9485"
Oct 31 22:19:36.189222 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:36.189205    4460 scope.go:122] "RemoveContainer" containerID="042625313bc5ef40a0e4e1d1edf6965ed38f3a1ca719a93ef72c1422edacb6e2"
Oct 31 22:19:36.198274 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:36.198252    4460 scope.go:122] "RemoveContainer" containerID="4e95ab6cd2ec240baa1145ea1a369aee448ed29a08a3733be044b148403b9485"
Oct 31 22:19:36.198539 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:19:36.198511    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e95ab6cd2ec240baa1145ea1a369aee448ed29a08a3733be044b148403b9485\": container with ID starting with 4e95ab6cd2ec240baa1145ea1a369aee448ed29a08a3733be044b148403b9485 not found: ID does not exist" containerID="4e95ab6cd2ec240baa1145ea1a369aee448ed29a08a3733be044b148403b9485"
Oct 31 22:19:36.198604 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:36.198549    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e95ab6cd2ec240baa1145ea1a369aee448ed29a08a3733be044b148403b9485"} err="failed to get container status \"4e95ab6cd2ec240baa1145ea1a369aee448ed29a08a3733be044b148403b9485\": rpc error: code = NotFound desc = could not find container \"4e95ab6cd2ec240baa1145ea1a369aee448ed29a08a3733be044b148403b9485\": container with ID starting with 4e95ab6cd2ec240baa1145ea1a369aee448ed29a08a3733be044b148403b9485 not found: ID does not exist"
Oct 31 22:19:36.198604 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:36.198568    4460 scope.go:122] "RemoveContainer" containerID="042625313bc5ef40a0e4e1d1edf6965ed38f3a1ca719a93ef72c1422edacb6e2"
Oct 31 22:19:36.198826 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:19:36.198805    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"042625313bc5ef40a0e4e1d1edf6965ed38f3a1ca719a93ef72c1422edacb6e2\": container with ID starting with 042625313bc5ef40a0e4e1d1edf6965ed38f3a1ca719a93ef72c1422edacb6e2 not found: ID does not exist" containerID="042625313bc5ef40a0e4e1d1edf6965ed38f3a1ca719a93ef72c1422edacb6e2"
Oct 31 22:19:36.198887 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:36.198835    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"042625313bc5ef40a0e4e1d1edf6965ed38f3a1ca719a93ef72c1422edacb6e2"} err="failed to get container status \"042625313bc5ef40a0e4e1d1edf6965ed38f3a1ca719a93ef72c1422edacb6e2\": rpc error: code = NotFound desc = could not find container \"042625313bc5ef40a0e4e1d1edf6965ed38f3a1ca719a93ef72c1422edacb6e2\": container with ID starting with 042625313bc5ef40a0e4e1d1edf6965ed38f3a1ca719a93ef72c1422edacb6e2 not found: ID does not exist"
Oct 31 22:19:36.224357 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:36.224333    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2b6a669d-e3cb-405b-a1c4-6fdfceb78990" path="/var/lib/kubelet/pods/2b6a669d-e3cb-405b-a1c4-6fdfceb78990/volumes"
Oct 31 22:19:36.236661 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:36.236639    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"]
Oct 31 22:19:36.250220 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:36.250198    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-5c54fc-openshift-default-6bcc667c45-q9cjv"]
Oct 31 22:19:36.933514 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:36.933473    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5c54fc-7467846795-qndhq" podUID="b7d48a8c-34fc-4974-8cc6-1346a27fcc65" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:19:37.105055 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:37.105019    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:37.862881 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:37.862837    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 22:19:37.969605 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:37.969578    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 22:19:38.124158 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.124125    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3f60d565-50f1-4b9d-959a-014be6e49128-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f60d565-50f1-4b9d-959a-014be6e49128-istiod-ca-cert\") pod \"3f60d565-50f1-4b9d-959a-014be6e49128\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") "
Oct 31 22:19:38.124365 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.124172    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-workload-socket\") pod \"3f60d565-50f1-4b9d-959a-014be6e49128\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") "
Oct 31 22:19:38.124365 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.124206    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3f60d565-50f1-4b9d-959a-014be6e49128-istio-token\" (UniqueName: \"kubernetes.io/projected/3f60d565-50f1-4b9d-959a-014be6e49128-istio-token\") pod \"3f60d565-50f1-4b9d-959a-014be6e49128\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") "
Oct 31 22:19:38.124365 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.124227    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3f60d565-50f1-4b9d-959a-014be6e49128-kube-api-access-s85wb\" (UniqueName: \"kubernetes.io/projected/3f60d565-50f1-4b9d-959a-014be6e49128-kube-api-access-s85wb\") pod \"3f60d565-50f1-4b9d-959a-014be6e49128\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") "
Oct 31 22:19:38.124365 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.124312    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-workload-certs\") pod \"3f60d565-50f1-4b9d-959a-014be6e49128\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") "
Oct 31 22:19:38.124605 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.124384    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-credential-socket\") pod \"3f60d565-50f1-4b9d-959a-014be6e49128\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") "
Oct 31 22:19:38.124605 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.124415    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-istio-envoy\") pod \"3f60d565-50f1-4b9d-959a-014be6e49128\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") "
Oct 31 22:19:38.124605 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.124493    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-istio-data\") pod \"3f60d565-50f1-4b9d-959a-014be6e49128\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") "
Oct 31 22:19:38.124605 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.124549    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3f60d565-50f1-4b9d-959a-014be6e49128-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f60d565-50f1-4b9d-959a-014be6e49128-istio-podinfo\") pod \"3f60d565-50f1-4b9d-959a-014be6e49128\" (UID: \"3f60d565-50f1-4b9d-959a-014be6e49128\") "
Oct 31 22:19:38.124605 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.124546    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-workload-socket" pod "3f60d565-50f1-4b9d-959a-014be6e49128" (UID: "3f60d565-50f1-4b9d-959a-014be6e49128"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:19:38.124605 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.124546    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f60d565-50f1-4b9d-959a-014be6e49128-istiod-ca-cert" pod "3f60d565-50f1-4b9d-959a-014be6e49128" (UID: "3f60d565-50f1-4b9d-959a-014be6e49128"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:19:38.124962 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.124642    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-workload-certs" pod "3f60d565-50f1-4b9d-959a-014be6e49128" (UID: "3f60d565-50f1-4b9d-959a-014be6e49128"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:19:38.124962 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.124690    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-credential-socket" pod "3f60d565-50f1-4b9d-959a-014be6e49128" (UID: "3f60d565-50f1-4b9d-959a-014be6e49128"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:19:38.124962 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.124825    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-istio-data" pod "3f60d565-50f1-4b9d-959a-014be6e49128" (UID: "3f60d565-50f1-4b9d-959a-014be6e49128"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:19:38.125131 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.125090    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f60d565-50f1-4b9d-959a-014be6e49128-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:38.125131 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.125115    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:38.125213 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.125132    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:38.125213 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.125149    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:38.125213 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.125164    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:38.126705 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.126681    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3f60d565-50f1-4b9d-959a-014be6e49128-istio-podinfo" pod "3f60d565-50f1-4b9d-959a-014be6e49128" (UID: "3f60d565-50f1-4b9d-959a-014be6e49128"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 22:19:38.126792 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.126700    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f60d565-50f1-4b9d-959a-014be6e49128-kube-api-access-s85wb" pod "3f60d565-50f1-4b9d-959a-014be6e49128" (UID: "3f60d565-50f1-4b9d-959a-014be6e49128"). InnerVolumeSpecName "kube-api-access-s85wb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:19:38.127088 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.127069    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f60d565-50f1-4b9d-959a-014be6e49128-istio-token" pod "3f60d565-50f1-4b9d-959a-014be6e49128" (UID: "3f60d565-50f1-4b9d-959a-014be6e49128"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:19:38.127148 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.127127    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-istio-envoy" pod "3f60d565-50f1-4b9d-959a-014be6e49128" (UID: "3f60d565-50f1-4b9d-959a-014be6e49128"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:19:38.193004 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.192970    4460 generic.go:356] "Generic (PLEG): container finished" podID="3f60d565-50f1-4b9d-959a-014be6e49128" containerID="b3d79e8dcda1d1ffba9095e19db1120efec82491a5488ddb2692667aedcb0ade" exitCode=0
Oct 31 22:19:38.193190 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.193040    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"
Oct 31 22:19:38.193190 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.193059    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk" event={"ID":"3f60d565-50f1-4b9d-959a-014be6e49128","Type":"ContainerDied","Data":"b3d79e8dcda1d1ffba9095e19db1120efec82491a5488ddb2692667aedcb0ade"}
Oct 31 22:19:38.193190 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.193119    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk" event={"ID":"3f60d565-50f1-4b9d-959a-014be6e49128","Type":"ContainerDied","Data":"0fce095fd23fd5b4dcc980c0481af897bf1664a19edc67e509e5c796ae47d9b3"}
Oct 31 22:19:38.193190 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.193145    4460 scope.go:122] "RemoveContainer" containerID="b3d79e8dcda1d1ffba9095e19db1120efec82491a5488ddb2692667aedcb0ade"
Oct 31 22:19:38.203547 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.203528    4460 scope.go:122] "RemoveContainer" containerID="b3d79e8dcda1d1ffba9095e19db1120efec82491a5488ddb2692667aedcb0ade"
Oct 31 22:19:38.203851 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:19:38.203825    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3d79e8dcda1d1ffba9095e19db1120efec82491a5488ddb2692667aedcb0ade\": container with ID starting with b3d79e8dcda1d1ffba9095e19db1120efec82491a5488ddb2692667aedcb0ade not found: ID does not exist" containerID="b3d79e8dcda1d1ffba9095e19db1120efec82491a5488ddb2692667aedcb0ade"
Oct 31 22:19:38.203979 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.203856    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3d79e8dcda1d1ffba9095e19db1120efec82491a5488ddb2692667aedcb0ade"} err="failed to get container status \"b3d79e8dcda1d1ffba9095e19db1120efec82491a5488ddb2692667aedcb0ade\": rpc error: code = NotFound desc = could not find container \"b3d79e8dcda1d1ffba9095e19db1120efec82491a5488ddb2692667aedcb0ade\": container with ID starting with b3d79e8dcda1d1ffba9095e19db1120efec82491a5488ddb2692667aedcb0ade not found: ID does not exist"
Oct 31 22:19:38.228255 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.228229    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f60d565-50f1-4b9d-959a-014be6e49128-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:38.228255 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.228253    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s85wb\" (UniqueName: \"kubernetes.io/projected/3f60d565-50f1-4b9d-959a-014be6e49128-kube-api-access-s85wb\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:38.228488 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.228270    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f60d565-50f1-4b9d-959a-014be6e49128-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:38.228488 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.228281    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f60d565-50f1-4b9d-959a-014be6e49128-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:38.232347 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.232312    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="390c66e4-91b0-4110-92c3-fd754193823d" path="/var/lib/kubelet/pods/390c66e4-91b0-4110-92c3-fd754193823d/volumes"
Oct 31 22:19:38.252220 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.252190    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"]
Oct 31 22:19:38.256162 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.256139    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-cc45a7-openshift-default-55cdb6f7d7-mxsbk"]
Oct 31 22:19:38.321398 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.321364    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-fb32d4-b4fd9fb87-l6tzn"]
Oct 31 22:19:38.464295 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.464207    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:38.577904 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.577859    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:38.711697 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:38.711659    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:40.214133 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:40.214087    4460 generic.go:356] "Generic (PLEG): container finished" podID="b7d48a8c-34fc-4974-8cc6-1346a27fcc65" containerID="676db78d3a27c5c5a346f580ff3bce56d0e4bf6e3acd048cbfd30ac114d09951" exitCode=0
Oct 31 22:19:40.214637 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:40.214231    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5c54fc-7467846795-qndhq" event={"ID":"b7d48a8c-34fc-4974-8cc6-1346a27fcc65","Type":"ContainerDied","Data":"676db78d3a27c5c5a346f580ff3bce56d0e4bf6e3acd048cbfd30ac114d09951"}
Oct 31 22:19:40.225784 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:40.225755    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:40.228072 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:40.228042    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3f60d565-50f1-4b9d-959a-014be6e49128" path="/var/lib/kubelet/pods/3f60d565-50f1-4b9d-959a-014be6e49128/volumes"
Oct 31 22:19:40.281535 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:40.281499    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5c54fc-7467846795-qndhq"
Oct 31 22:19:40.391780 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:40.391746    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:40.452770 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:40.452732    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b7d48a8c-34fc-4974-8cc6-1346a27fcc65-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7d48a8c-34fc-4974-8cc6-1346a27fcc65-plugins-config-volume\") pod \"b7d48a8c-34fc-4974-8cc6-1346a27fcc65\" (UID: \"b7d48a8c-34fc-4974-8cc6-1346a27fcc65\") "
Oct 31 22:19:40.452974 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:40.452912    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b7d48a8c-34fc-4974-8cc6-1346a27fcc65-kube-api-access-tsgmm\" (UniqueName: \"kubernetes.io/projected/b7d48a8c-34fc-4974-8cc6-1346a27fcc65-kube-api-access-tsgmm\") pod \"b7d48a8c-34fc-4974-8cc6-1346a27fcc65\" (UID: \"b7d48a8c-34fc-4974-8cc6-1346a27fcc65\") "
Oct 31 22:19:40.453228 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:40.453198    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7d48a8c-34fc-4974-8cc6-1346a27fcc65-plugins-config-volume" pod "b7d48a8c-34fc-4974-8cc6-1346a27fcc65" (UID: "b7d48a8c-34fc-4974-8cc6-1346a27fcc65"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:19:40.453334 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:40.453319    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7d48a8c-34fc-4974-8cc6-1346a27fcc65-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:40.455082 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:40.455058    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7d48a8c-34fc-4974-8cc6-1346a27fcc65-kube-api-access-tsgmm" pod "b7d48a8c-34fc-4974-8cc6-1346a27fcc65" (UID: "b7d48a8c-34fc-4974-8cc6-1346a27fcc65"). InnerVolumeSpecName "kube-api-access-tsgmm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:19:40.504950 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:40.504844    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:40.558578 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:40.558535    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tsgmm\" (UniqueName: \"kubernetes.io/projected/b7d48a8c-34fc-4974-8cc6-1346a27fcc65-kube-api-access-tsgmm\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:40.615921 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:40.615882    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:40.729141 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:40.729100    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:40.874993 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:40.874925    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:41.221646 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:41.221560    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5c54fc-7467846795-qndhq"
Oct 31 22:19:41.221646 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:41.221590    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-5c54fc-7467846795-qndhq" event={"ID":"b7d48a8c-34fc-4974-8cc6-1346a27fcc65","Type":"ContainerDied","Data":"3230ed0a1685d69410467e9f88690da086dba0adfa7a956f8b8f139655249e71"}
Oct 31 22:19:41.221646 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:41.221646    4460 scope.go:122] "RemoveContainer" containerID="676db78d3a27c5c5a346f580ff3bce56d0e4bf6e3acd048cbfd30ac114d09951"
Oct 31 22:19:41.276995 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:41.276932    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5c54fc-7467846795-qndhq"]
Oct 31 22:19:41.291299 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:41.291264    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-5c54fc-7467846795-qndhq"]
Oct 31 22:19:41.427580 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:41.427535    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-943963-7568c9dd8c-99fk7"]
Oct 31 22:19:41.461032 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:41.460996    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-943963-688574d587-7wwhp"]
Oct 31 22:19:41.571712 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:41.571675    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:41.681377 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:41.681338    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:41.796692 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:41.796657    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:42.227537 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:42.227502    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b7d48a8c-34fc-4974-8cc6-1346a27fcc65" path="/var/lib/kubelet/pods/b7d48a8c-34fc-4974-8cc6-1346a27fcc65/volumes"
Oct 31 22:19:42.299610 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:42.299569    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:42.408526 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:42.408492    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:42.912815 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:42.912730    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:43.321646 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:43.321589    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fb32d4-b4fd9fb87-l6tzn" podUID="982141b1-3b70-4767-9442-264f8920c0b5" containerName="payload-processing" containerID="cri-o://8124a6b000290299b000b239acbbf112d08f06b4383471f166faafd1f2350a8b" gracePeriod=25
Oct 31 22:19:44.179990 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:44.179907    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:45.904233 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:45.904196    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:46.428438 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:46.428393    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-943963-7568c9dd8c-99fk7" podUID="406f2702-7e47-4467-9f9c-a41f8f9ea53d" containerName="payload-processing" containerID="cri-o://6ca7c94dc2bba91d807925df328df4cccc3fa917dda36a58ed9e9db458bc7637" gracePeriod=25
Oct 31 22:19:46.462115 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:46.462065    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-943963-688574d587-7wwhp" podUID="ebe9a6f2-adf2-4256-bbea-963048f4a0ad" containerName="payload-pre-processing" containerID="cri-o://288a53ef1d3ce07122f7a7967d0112323073357200626ef55b48d1031ff0aaf5" gracePeriod=25
Oct 31 22:19:47.548266 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:47.548223    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:48.728290 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:48.728254    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:48.887088 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:48.887055    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx"]
Oct 31 22:19:48.887342 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:48.887293    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx" podUID="a8c618bb-9eca-482e-8d17-f0f7a1ffd12c" containerName="maas-api" containerID="cri-o://caf95d698a24d76171c54011ee3d8fe7c403fe3e41b1a4d79ab3f2e8815d2bb1" gracePeriod=30
Oct 31 22:19:48.951001 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:48.950935    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:49.148908 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:49.148884    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx"
Oct 31 22:19:49.253177 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:49.253100    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a8c618bb-9eca-482e-8d17-f0f7a1ffd12c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8c618bb-9eca-482e-8d17-f0f7a1ffd12c-metrics-certs\") pod \"a8c618bb-9eca-482e-8d17-f0f7a1ffd12c\" (UID: \"a8c618bb-9eca-482e-8d17-f0f7a1ffd12c\") "
Oct 31 22:19:49.253329 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:49.253268    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8c618bb-9eca-482e-8d17-f0f7a1ffd12c-kube-api-access-xlh8r\" (UniqueName: \"kubernetes.io/projected/a8c618bb-9eca-482e-8d17-f0f7a1ffd12c-kube-api-access-xlh8r\") pod \"a8c618bb-9eca-482e-8d17-f0f7a1ffd12c\" (UID: \"a8c618bb-9eca-482e-8d17-f0f7a1ffd12c\") "
Oct 31 22:19:49.253329 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:49.253290    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a8c618bb-9eca-482e-8d17-f0f7a1ffd12c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8c618bb-9eca-482e-8d17-f0f7a1ffd12c-maas-api-tls\") pod \"a8c618bb-9eca-482e-8d17-f0f7a1ffd12c\" (UID: \"a8c618bb-9eca-482e-8d17-f0f7a1ffd12c\") "
Oct 31 22:19:49.253329 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:49.253315    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a8c618bb-9eca-482e-8d17-f0f7a1ffd12c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8c618bb-9eca-482e-8d17-f0f7a1ffd12c-openshift-service-ca\") pod \"a8c618bb-9eca-482e-8d17-f0f7a1ffd12c\" (UID: \"a8c618bb-9eca-482e-8d17-f0f7a1ffd12c\") "
Oct 31 22:19:49.253822 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:49.253786    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8c618bb-9eca-482e-8d17-f0f7a1ffd12c-openshift-service-ca" pod "a8c618bb-9eca-482e-8d17-f0f7a1ffd12c" (UID: "a8c618bb-9eca-482e-8d17-f0f7a1ffd12c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:19:49.255431 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:49.255397    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8c618bb-9eca-482e-8d17-f0f7a1ffd12c-kube-api-access-xlh8r" pod "a8c618bb-9eca-482e-8d17-f0f7a1ffd12c" (UID: "a8c618bb-9eca-482e-8d17-f0f7a1ffd12c"). InnerVolumeSpecName "kube-api-access-xlh8r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:19:49.255526 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:49.255475    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8c618bb-9eca-482e-8d17-f0f7a1ffd12c-metrics-certs" pod "a8c618bb-9eca-482e-8d17-f0f7a1ffd12c" (UID: "a8c618bb-9eca-482e-8d17-f0f7a1ffd12c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 22:19:49.255526 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:49.255498    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8c618bb-9eca-482e-8d17-f0f7a1ffd12c-maas-api-tls" pod "a8c618bb-9eca-482e-8d17-f0f7a1ffd12c" (UID: "a8c618bb-9eca-482e-8d17-f0f7a1ffd12c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 22:19:49.272803 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:49.272777    4460 generic.go:356] "Generic (PLEG): container finished" podID="a8c618bb-9eca-482e-8d17-f0f7a1ffd12c" containerID="caf95d698a24d76171c54011ee3d8fe7c403fe3e41b1a4d79ab3f2e8815d2bb1" exitCode=0
Oct 31 22:19:49.272927 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:49.272816    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx"
Oct 31 22:19:49.272927 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:49.272856    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx" event={"ID":"a8c618bb-9eca-482e-8d17-f0f7a1ffd12c","Type":"ContainerDied","Data":"caf95d698a24d76171c54011ee3d8fe7c403fe3e41b1a4d79ab3f2e8815d2bb1"}
Oct 31 22:19:49.272927 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:49.272922    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx" event={"ID":"a8c618bb-9eca-482e-8d17-f0f7a1ffd12c","Type":"ContainerDied","Data":"e59572b0b968801ec4546fbf858134b5c19dc7ba85e28f58757e0e90536a1bbd"}
Oct 31 22:19:49.273121 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:49.272969    4460 scope.go:122] "RemoveContainer" containerID="caf95d698a24d76171c54011ee3d8fe7c403fe3e41b1a4d79ab3f2e8815d2bb1"
Oct 31 22:19:49.283166 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:49.283142    4460 scope.go:122] "RemoveContainer" containerID="caf95d698a24d76171c54011ee3d8fe7c403fe3e41b1a4d79ab3f2e8815d2bb1"
Oct 31 22:19:49.283467 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:19:49.283445    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"caf95d698a24d76171c54011ee3d8fe7c403fe3e41b1a4d79ab3f2e8815d2bb1\": container with ID starting with caf95d698a24d76171c54011ee3d8fe7c403fe3e41b1a4d79ab3f2e8815d2bb1 not found: ID does not exist" containerID="caf95d698a24d76171c54011ee3d8fe7c403fe3e41b1a4d79ab3f2e8815d2bb1"
Oct 31 22:19:49.283522 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:49.283474    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caf95d698a24d76171c54011ee3d8fe7c403fe3e41b1a4d79ab3f2e8815d2bb1"} err="failed to get container status \"caf95d698a24d76171c54011ee3d8fe7c403fe3e41b1a4d79ab3f2e8815d2bb1\": rpc error: code = NotFound desc = could not find container \"caf95d698a24d76171c54011ee3d8fe7c403fe3e41b1a4d79ab3f2e8815d2bb1\": container with ID starting with caf95d698a24d76171c54011ee3d8fe7c403fe3e41b1a4d79ab3f2e8815d2bb1 not found: ID does not exist"
Oct 31 22:19:49.347039 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:49.341268    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx"]
Oct 31 22:19:49.347039 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:49.345037    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-943963-6474bdb6cc-xmxvx"]
Oct 31 22:19:49.355073 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:49.355039    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlh8r\" (UniqueName: \"kubernetes.io/projected/a8c618bb-9eca-482e-8d17-f0f7a1ffd12c-kube-api-access-xlh8r\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:49.355073 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:49.355069    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8c618bb-9eca-482e-8d17-f0f7a1ffd12c-maas-api-tls\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:49.355073 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:49.355080    4460 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8c618bb-9eca-482e-8d17-f0f7a1ffd12c-openshift-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:49.355320 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:49.355088    4460 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8c618bb-9eca-482e-8d17-f0f7a1ffd12c-metrics-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:19:49.391697 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:49.391648    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:49.498398 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:49.498362    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:49.607280 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:49.607248    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:49.751704 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:49.751666    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:49.858300 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:49.858213    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:49.978594 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:49.978555    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:50.091754 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:50.091715    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:50.226777 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:50.226698    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a8c618bb-9eca-482e-8d17-f0f7a1ffd12c" path="/var/lib/kubelet/pods/a8c618bb-9eca-482e-8d17-f0f7a1ffd12c/volumes"
Oct 31 22:19:50.547484 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:50.547443    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:51.346637 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:51.346599    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:51.452356 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:19:51.452317    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:19:51.531638 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:51.531602    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:52.913584 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:52.913545    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:52.936965 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:52.936913    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-943963-688574d587-7wwhp" podUID="ebe9a6f2-adf2-4256-bbea-963048f4a0ad" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:19:52.936965 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:52.936954    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-943963-7568c9dd8c-99fk7" podUID="406f2702-7e47-4467-9f9c-a41f8f9ea53d" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:19:53.163412 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:53.163366    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fb32d4-b4fd9fb87-l6tzn" podUID="982141b1-3b70-4767-9442-264f8920c0b5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:19:53.663955 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:53.663889    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 22:19:54.496653 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:54.496605    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:55.513620 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:55.513587    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"]
Oct 31 22:19:55.528056 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:55.528024    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"]
Oct 31 22:19:55.655487 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:55.655445    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:56.128473 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:56.128428    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:56.836438 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:56.836404    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:56.943117 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:56.943082    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:57.121169 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:57.121081    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"]
Oct 31 22:19:57.121411 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:57.121382    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf" podUID="b07cb206-7fc8-42a4-9086-dac5ce1d7019" containerName="istio-proxy" containerID="cri-o://4f67c0688eda6f578a35f1cfa5c5fc9795bfa35c473f78180c0933a13418ebf3" gracePeriod=30
Oct 31 22:19:57.155551 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:57.155518    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:57.778049 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:57.778016    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:19:59.508512 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:19:59.508472    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:01.372037 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:01.371990    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:01.484088 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:01.484050    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:02.353501 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:02.353471    4460 generic.go:356] "Generic (PLEG): container finished" podID="b07cb206-7fc8-42a4-9086-dac5ce1d7019" containerID="4f67c0688eda6f578a35f1cfa5c5fc9795bfa35c473f78180c0933a13418ebf3" exitCode=0
Oct 31 22:20:02.353681 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:02.353497    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf" event={"ID":"b07cb206-7fc8-42a4-9086-dac5ce1d7019","Type":"ContainerDied","Data":"4f67c0688eda6f578a35f1cfa5c5fc9795bfa35c473f78180c0933a13418ebf3"}
Oct 31 22:20:02.353681 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:02.353565    4460 scope.go:122] "RemoveContainer" containerID="03dcb7cdafbd70b4db5bada1f9d3ca6ab727fec4ccd5325598f97683283b6967"
Oct 31 22:20:02.391893 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:02.391844    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 22:20:02.500130 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:02.500101    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-credential-socket\") pod \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") "
Oct 31 22:20:02.500285 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:02.500162    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b07cb206-7fc8-42a4-9086-dac5ce1d7019-kube-api-access-ccfnq\" (UniqueName: \"kubernetes.io/projected/b07cb206-7fc8-42a4-9086-dac5ce1d7019-kube-api-access-ccfnq\") pod \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") "
Oct 31 22:20:02.500285 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:02.500203    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istio-envoy\") pod \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") "
Oct 31 22:20:02.500285 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:02.500229    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-workload-certs\") pod \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") "
Oct 31 22:20:02.500285 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:02.500267    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istio-token\" (UniqueName: \"kubernetes.io/projected/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istio-token\") pod \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") "
Oct 31 22:20:02.500486 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:02.500299    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istiod-ca-cert\") pod \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") "
Oct 31 22:20:02.500486 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:02.500382    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istio-podinfo\") pod \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") "
Oct 31 22:20:02.500486 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:02.500464    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-workload-socket\") pod \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") "
Oct 31 22:20:02.500667 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:02.500540    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istio-data\") pod \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\" (UID: \"b07cb206-7fc8-42a4-9086-dac5ce1d7019\") "
Oct 31 22:20:02.500667 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:02.500457    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-credential-socket" pod "b07cb206-7fc8-42a4-9086-dac5ce1d7019" (UID: "b07cb206-7fc8-42a4-9086-dac5ce1d7019"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:20:02.500667 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:02.500571    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-workload-certs" pod "b07cb206-7fc8-42a4-9086-dac5ce1d7019" (UID: "b07cb206-7fc8-42a4-9086-dac5ce1d7019"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:20:02.500834 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:02.500753    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istiod-ca-cert" pod "b07cb206-7fc8-42a4-9086-dac5ce1d7019" (UID: "b07cb206-7fc8-42a4-9086-dac5ce1d7019"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:20:02.500925 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:02.500899    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istio-data" pod "b07cb206-7fc8-42a4-9086-dac5ce1d7019" (UID: "b07cb206-7fc8-42a4-9086-dac5ce1d7019"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:20:02.501063 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:02.500991    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-workload-socket" pod "b07cb206-7fc8-42a4-9086-dac5ce1d7019" (UID: "b07cb206-7fc8-42a4-9086-dac5ce1d7019"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:20:02.501176 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:02.501156    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:02.501225 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:02.501184    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:02.501225 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:02.501201    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:02.501225 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:02.501218    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:02.501348 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:02.501232    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:02.502528 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:02.502493    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b07cb206-7fc8-42a4-9086-dac5ce1d7019-kube-api-access-ccfnq" pod "b07cb206-7fc8-42a4-9086-dac5ce1d7019" (UID: "b07cb206-7fc8-42a4-9086-dac5ce1d7019"). InnerVolumeSpecName "kube-api-access-ccfnq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:20:02.502691 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:02.502675    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istio-token" pod "b07cb206-7fc8-42a4-9086-dac5ce1d7019" (UID: "b07cb206-7fc8-42a4-9086-dac5ce1d7019"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:20:02.502860 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:02.502837    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istio-envoy" pod "b07cb206-7fc8-42a4-9086-dac5ce1d7019" (UID: "b07cb206-7fc8-42a4-9086-dac5ce1d7019"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:20:02.502998 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:02.502981    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istio-podinfo" pod "b07cb206-7fc8-42a4-9086-dac5ce1d7019" (UID: "b07cb206-7fc8-42a4-9086-dac5ce1d7019"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 22:20:02.601786 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:02.601755    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ccfnq\" (UniqueName: \"kubernetes.io/projected/b07cb206-7fc8-42a4-9086-dac5ce1d7019-kube-api-access-ccfnq\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:02.601786 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:02.601785    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:02.601786 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:02.601794    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:02.602044 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:02.601802    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b07cb206-7fc8-42a4-9086-dac5ce1d7019-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:02.936846 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:02.936805    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-943963-688574d587-7wwhp" podUID="ebe9a6f2-adf2-4256-bbea-963048f4a0ad" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:20:02.936846 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:02.936832    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-943963-7568c9dd8c-99fk7" podUID="406f2702-7e47-4467-9f9c-a41f8f9ea53d" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:20:03.028732 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:03.028699    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:03.162928 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:03.162887    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fb32d4-b4fd9fb87-l6tzn" podUID="982141b1-3b70-4767-9442-264f8920c0b5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:20:03.360643 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:03.360616    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"
Oct 31 22:20:03.360764 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:03.360663    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf" event={"ID":"b07cb206-7fc8-42a4-9086-dac5ce1d7019","Type":"ContainerDied","Data":"d245ddbabfbd55465cd0cf942e235c83b5a41b8075f2f89047ac729d493740d6"}
Oct 31 22:20:03.360764 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:03.360729    4460 scope.go:122] "RemoveContainer" containerID="4f67c0688eda6f578a35f1cfa5c5fc9795bfa35c473f78180c0933a13418ebf3"
Oct 31 22:20:03.427380 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:03.427351    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"]
Oct 31 22:20:03.435049 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:03.435022    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-943963-openshift-default-68fddd7bb5-44fhf"]
Oct 31 22:20:03.589631 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:03.589568    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fb32d4-b4fd9fb87-l6tzn"
Oct 31 22:20:03.711011 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:03.710975    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/982141b1-3b70-4767-9442-264f8920c0b5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/982141b1-3b70-4767-9442-264f8920c0b5-plugins-config-volume\") pod \"982141b1-3b70-4767-9442-264f8920c0b5\" (UID: \"982141b1-3b70-4767-9442-264f8920c0b5\") "
Oct 31 22:20:03.711207 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:03.711091    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/982141b1-3b70-4767-9442-264f8920c0b5-kube-api-access-s7gjt\" (UniqueName: \"kubernetes.io/projected/982141b1-3b70-4767-9442-264f8920c0b5-kube-api-access-s7gjt\") pod \"982141b1-3b70-4767-9442-264f8920c0b5\" (UID: \"982141b1-3b70-4767-9442-264f8920c0b5\") "
Oct 31 22:20:03.711660 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:03.711387    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/982141b1-3b70-4767-9442-264f8920c0b5-plugins-config-volume" pod "982141b1-3b70-4767-9442-264f8920c0b5" (UID: "982141b1-3b70-4767-9442-264f8920c0b5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:20:03.711660 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:03.711636    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/982141b1-3b70-4767-9442-264f8920c0b5-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:03.713842 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:03.713811    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/982141b1-3b70-4767-9442-264f8920c0b5-kube-api-access-s7gjt" pod "982141b1-3b70-4767-9442-264f8920c0b5" (UID: "982141b1-3b70-4767-9442-264f8920c0b5"). InnerVolumeSpecName "kube-api-access-s7gjt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:20:03.813206 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:03.813172    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7gjt\" (UniqueName: \"kubernetes.io/projected/982141b1-3b70-4767-9442-264f8920c0b5-kube-api-access-s7gjt\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:04.224049 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:04.224008    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b07cb206-7fc8-42a4-9086-dac5ce1d7019" path="/var/lib/kubelet/pods/b07cb206-7fc8-42a4-9086-dac5ce1d7019/volumes"
Oct 31 22:20:04.368142 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:04.368105    4460 generic.go:356] "Generic (PLEG): container finished" podID="982141b1-3b70-4767-9442-264f8920c0b5" containerID="8124a6b000290299b000b239acbbf112d08f06b4383471f166faafd1f2350a8b" exitCode=0
Oct 31 22:20:04.368278 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:04.368146    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fb32d4-b4fd9fb87-l6tzn"
Oct 31 22:20:04.368278 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:04.368175    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fb32d4-b4fd9fb87-l6tzn" event={"ID":"982141b1-3b70-4767-9442-264f8920c0b5","Type":"ContainerDied","Data":"8124a6b000290299b000b239acbbf112d08f06b4383471f166faafd1f2350a8b"}
Oct 31 22:20:04.368278 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:04.368224    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fb32d4-b4fd9fb87-l6tzn" event={"ID":"982141b1-3b70-4767-9442-264f8920c0b5","Type":"ContainerDied","Data":"f88dd3eeb4aad082e9ac1fb5d8ebcd0197aa17d8c0c714bf069bdf20aedb843e"}
Oct 31 22:20:04.368278 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:04.368250    4460 scope.go:122] "RemoveContainer" containerID="8124a6b000290299b000b239acbbf112d08f06b4383471f166faafd1f2350a8b"
Oct 31 22:20:04.377520 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:04.377499    4460 scope.go:122] "RemoveContainer" containerID="8124a6b000290299b000b239acbbf112d08f06b4383471f166faafd1f2350a8b"
Oct 31 22:20:04.377812 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:20:04.377785    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8124a6b000290299b000b239acbbf112d08f06b4383471f166faafd1f2350a8b\": container with ID starting with 8124a6b000290299b000b239acbbf112d08f06b4383471f166faafd1f2350a8b not found: ID does not exist" containerID="8124a6b000290299b000b239acbbf112d08f06b4383471f166faafd1f2350a8b"
Oct 31 22:20:04.377870 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:04.377819    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8124a6b000290299b000b239acbbf112d08f06b4383471f166faafd1f2350a8b"} err="failed to get container status \"8124a6b000290299b000b239acbbf112d08f06b4383471f166faafd1f2350a8b\": rpc error: code = NotFound desc = could not find container \"8124a6b000290299b000b239acbbf112d08f06b4383471f166faafd1f2350a8b\": container with ID starting with 8124a6b000290299b000b239acbbf112d08f06b4383471f166faafd1f2350a8b not found: ID does not exist"
Oct 31 22:20:04.424821 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:04.424799    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-fb32d4-b4fd9fb87-l6tzn"]
Oct 31 22:20:04.438414 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:04.438393    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-fb32d4-b4fd9fb87-l6tzn"]
Oct 31 22:20:04.615745 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:04.615712    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:06.031102 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:06.031064    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cd6e81c8-th99j"]
Oct 31 22:20:06.044328 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:06.044292    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cd6e81c8-th99j"]
Oct 31 22:20:06.219444 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:06.219401    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:06.223523 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:06.223496    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4b94e7af-e6fa-434b-b1ac-3069e1d6953e" path="/var/lib/kubelet/pods/4b94e7af-e6fa-434b-b1ac-3069e1d6953e/volumes"
Oct 31 22:20:06.223897 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:06.223883    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="982141b1-3b70-4767-9442-264f8920c0b5" path="/var/lib/kubelet/pods/982141b1-3b70-4767-9442-264f8920c0b5/volumes"
Oct 31 22:20:06.719195 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:06.719170    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-943963-7568c9dd8c-99fk7"
Oct 31 22:20:06.742341 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:06.742316    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-943963-688574d587-7wwhp"
Oct 31 22:20:06.840028 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:06.839996    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ebe9a6f2-adf2-4256-bbea-963048f4a0ad-kube-api-access-wfn59\" (UniqueName: \"kubernetes.io/projected/ebe9a6f2-adf2-4256-bbea-963048f4a0ad-kube-api-access-wfn59\") pod \"ebe9a6f2-adf2-4256-bbea-963048f4a0ad\" (UID: \"ebe9a6f2-adf2-4256-bbea-963048f4a0ad\") "
Oct 31 22:20:06.840215 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:06.840040    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/406f2702-7e47-4467-9f9c-a41f8f9ea53d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/406f2702-7e47-4467-9f9c-a41f8f9ea53d-plugins-config-volume\") pod \"406f2702-7e47-4467-9f9c-a41f8f9ea53d\" (UID: \"406f2702-7e47-4467-9f9c-a41f8f9ea53d\") "
Oct 31 22:20:06.840215 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:06.840121    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ebe9a6f2-adf2-4256-bbea-963048f4a0ad-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebe9a6f2-adf2-4256-bbea-963048f4a0ad-plugins-config-volume\") pod \"ebe9a6f2-adf2-4256-bbea-963048f4a0ad\" (UID: \"ebe9a6f2-adf2-4256-bbea-963048f4a0ad\") "
Oct 31 22:20:06.840344 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:06.840300    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/406f2702-7e47-4467-9f9c-a41f8f9ea53d-kube-api-access-8pv9d\" (UniqueName: \"kubernetes.io/projected/406f2702-7e47-4467-9f9c-a41f8f9ea53d-kube-api-access-8pv9d\") pod \"406f2702-7e47-4467-9f9c-a41f8f9ea53d\" (UID: \"406f2702-7e47-4467-9f9c-a41f8f9ea53d\") "
Oct 31 22:20:06.840528 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:06.840484    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebe9a6f2-adf2-4256-bbea-963048f4a0ad-plugins-config-volume" pod "ebe9a6f2-adf2-4256-bbea-963048f4a0ad" (UID: "ebe9a6f2-adf2-4256-bbea-963048f4a0ad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:20:06.840628 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:06.840544    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/406f2702-7e47-4467-9f9c-a41f8f9ea53d-plugins-config-volume" pod "406f2702-7e47-4467-9f9c-a41f8f9ea53d" (UID: "406f2702-7e47-4467-9f9c-a41f8f9ea53d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:20:06.840767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:06.840747    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/406f2702-7e47-4467-9f9c-a41f8f9ea53d-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:06.840836 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:06.840774    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebe9a6f2-adf2-4256-bbea-963048f4a0ad-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:06.842327 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:06.842305    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebe9a6f2-adf2-4256-bbea-963048f4a0ad-kube-api-access-wfn59" pod "ebe9a6f2-adf2-4256-bbea-963048f4a0ad" (UID: "ebe9a6f2-adf2-4256-bbea-963048f4a0ad"). InnerVolumeSpecName "kube-api-access-wfn59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:20:06.842417 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:06.842364    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/406f2702-7e47-4467-9f9c-a41f8f9ea53d-kube-api-access-8pv9d" pod "406f2702-7e47-4467-9f9c-a41f8f9ea53d" (UID: "406f2702-7e47-4467-9f9c-a41f8f9ea53d"). InnerVolumeSpecName "kube-api-access-8pv9d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:20:06.941904 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:06.941876    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pv9d\" (UniqueName: \"kubernetes.io/projected/406f2702-7e47-4467-9f9c-a41f8f9ea53d-kube-api-access-8pv9d\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:06.941904 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:06.941900    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfn59\" (UniqueName: \"kubernetes.io/projected/ebe9a6f2-adf2-4256-bbea-963048f4a0ad-kube-api-access-wfn59\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:07.385592 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:07.385557    4460 generic.go:356] "Generic (PLEG): container finished" podID="ebe9a6f2-adf2-4256-bbea-963048f4a0ad" containerID="288a53ef1d3ce07122f7a7967d0112323073357200626ef55b48d1031ff0aaf5" exitCode=0
Oct 31 22:20:07.386105 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:07.385603    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-943963-688574d587-7wwhp"
Oct 31 22:20:07.386105 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:07.385639    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-943963-688574d587-7wwhp" event={"ID":"ebe9a6f2-adf2-4256-bbea-963048f4a0ad","Type":"ContainerDied","Data":"288a53ef1d3ce07122f7a7967d0112323073357200626ef55b48d1031ff0aaf5"}
Oct 31 22:20:07.386105 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:07.385689    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-943963-688574d587-7wwhp" event={"ID":"ebe9a6f2-adf2-4256-bbea-963048f4a0ad","Type":"ContainerDied","Data":"55cfc8730b6540db5c214ebdf058190802730029d119aba8becadd86a02519b9"}
Oct 31 22:20:07.386105 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:07.385723    4460 scope.go:122] "RemoveContainer" containerID="288a53ef1d3ce07122f7a7967d0112323073357200626ef55b48d1031ff0aaf5"
Oct 31 22:20:07.387204 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:07.387176    4460 generic.go:356] "Generic (PLEG): container finished" podID="406f2702-7e47-4467-9f9c-a41f8f9ea53d" containerID="6ca7c94dc2bba91d807925df328df4cccc3fa917dda36a58ed9e9db458bc7637" exitCode=0
Oct 31 22:20:07.387341 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:07.387207    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-943963-7568c9dd8c-99fk7" event={"ID":"406f2702-7e47-4467-9f9c-a41f8f9ea53d","Type":"ContainerDied","Data":"6ca7c94dc2bba91d807925df328df4cccc3fa917dda36a58ed9e9db458bc7637"}
Oct 31 22:20:07.387341 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:07.387227    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-943963-7568c9dd8c-99fk7"
Oct 31 22:20:07.387341 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:07.387272    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-943963-7568c9dd8c-99fk7" event={"ID":"406f2702-7e47-4467-9f9c-a41f8f9ea53d","Type":"ContainerDied","Data":"0e941c269989753a6645778d57718432c9881dd7c29d32a5e4ea96c69d527d77"}
Oct 31 22:20:07.397117 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:07.397096    4460 scope.go:122] "RemoveContainer" containerID="288a53ef1d3ce07122f7a7967d0112323073357200626ef55b48d1031ff0aaf5"
Oct 31 22:20:07.397540 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:20:07.397394    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"288a53ef1d3ce07122f7a7967d0112323073357200626ef55b48d1031ff0aaf5\": container with ID starting with 288a53ef1d3ce07122f7a7967d0112323073357200626ef55b48d1031ff0aaf5 not found: ID does not exist" containerID="288a53ef1d3ce07122f7a7967d0112323073357200626ef55b48d1031ff0aaf5"
Oct 31 22:20:07.397540 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:07.397420    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"288a53ef1d3ce07122f7a7967d0112323073357200626ef55b48d1031ff0aaf5"} err="failed to get container status \"288a53ef1d3ce07122f7a7967d0112323073357200626ef55b48d1031ff0aaf5\": rpc error: code = NotFound desc = could not find container \"288a53ef1d3ce07122f7a7967d0112323073357200626ef55b48d1031ff0aaf5\": container with ID starting with 288a53ef1d3ce07122f7a7967d0112323073357200626ef55b48d1031ff0aaf5 not found: ID does not exist"
Oct 31 22:20:07.397540 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:07.397437    4460 scope.go:122] "RemoveContainer" containerID="6ca7c94dc2bba91d807925df328df4cccc3fa917dda36a58ed9e9db458bc7637"
Oct 31 22:20:07.405762 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:07.405744    4460 scope.go:122] "RemoveContainer" containerID="6ca7c94dc2bba91d807925df328df4cccc3fa917dda36a58ed9e9db458bc7637"
Oct 31 22:20:07.406026 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:20:07.406006    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ca7c94dc2bba91d807925df328df4cccc3fa917dda36a58ed9e9db458bc7637\": container with ID starting with 6ca7c94dc2bba91d807925df328df4cccc3fa917dda36a58ed9e9db458bc7637 not found: ID does not exist" containerID="6ca7c94dc2bba91d807925df328df4cccc3fa917dda36a58ed9e9db458bc7637"
Oct 31 22:20:07.406074 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:07.406034    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ca7c94dc2bba91d807925df328df4cccc3fa917dda36a58ed9e9db458bc7637"} err="failed to get container status \"6ca7c94dc2bba91d807925df328df4cccc3fa917dda36a58ed9e9db458bc7637\": rpc error: code = NotFound desc = could not find container \"6ca7c94dc2bba91d807925df328df4cccc3fa917dda36a58ed9e9db458bc7637\": container with ID starting with 6ca7c94dc2bba91d807925df328df4cccc3fa917dda36a58ed9e9db458bc7637 not found: ID does not exist"
Oct 31 22:20:07.441598 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:07.441568    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-943963-688574d587-7wwhp"]
Oct 31 22:20:07.455026 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:07.455005    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-943963-688574d587-7wwhp"]
Oct 31 22:20:07.484543 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:07.484516    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-943963-7568c9dd8c-99fk7"]
Oct 31 22:20:07.488589 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:07.488562    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-943963-7568c9dd8c-99fk7"]
Oct 31 22:20:07.819614 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:07.819574    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:08.156442 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:08.156341    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-56a6d4-6758676965-4l9dm"]
Oct 31 22:20:08.191712 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:08.191681    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-56a6d4-7769777f8c-2jddx"]
Oct 31 22:20:08.223464 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:08.223418    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="406f2702-7e47-4467-9f9c-a41f8f9ea53d" path="/var/lib/kubelet/pods/406f2702-7e47-4467-9f9c-a41f8f9ea53d/volumes"
Oct 31 22:20:08.223970 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:08.223918    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ebe9a6f2-adf2-4256-bbea-963048f4a0ad" path="/var/lib/kubelet/pods/ebe9a6f2-adf2-4256-bbea-963048f4a0ad/volumes"
Oct 31 22:20:08.326193 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:08.326155    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:08.437485 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:08.437395    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:09.514611 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:09.514579    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:10.534914 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.534875    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh" podUID="869ed893-352c-4f8f-a174-b42afdeb4de7" containerName="main" containerID="cri-o://e0817b2f9f16f4aec1b829e0d1daf6425a927c079717e4a035f854e4b64b4b46" gracePeriod=45
Oct 31 22:20:10.548355 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.548300    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747" podUID="d1078aa7-1947-45ce-b6bb-d6550ad61fcf" containerName="main" containerID="cri-o://af5b231520ddabdca0df6d92f164fe8c6fc8afa26d8ec336367e967e5f5d8350" gracePeriod=45
Oct 31 22:20:10.671898 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.671858    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh" podUID="869ed893-352c-4f8f-a174-b42afdeb4de7" containerName="main" probeResult="failure" output="Get \"https://10.133.0.77:8000/ready\": dial tcp 10.133.0.77:8000: connect: connection refused"
Oct 31 22:20:10.808357 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.808297    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 22:20:10.828643 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.828621    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 22:20:10.880683 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.880651    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/869ed893-352c-4f8f-a174-b42afdeb4de7-tls-certs\" (UniqueName: \"kubernetes.io/secret/869ed893-352c-4f8f-a174-b42afdeb4de7-tls-certs\") pod \"869ed893-352c-4f8f-a174-b42afdeb4de7\" (UID: \"869ed893-352c-4f8f-a174-b42afdeb4de7\") "
Oct 31 22:20:10.880868 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.880734    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/869ed893-352c-4f8f-a174-b42afdeb4de7-dshm\" (UniqueName: \"kubernetes.io/empty-dir/869ed893-352c-4f8f-a174-b42afdeb4de7-dshm\") pod \"869ed893-352c-4f8f-a174-b42afdeb4de7\" (UID: \"869ed893-352c-4f8f-a174-b42afdeb4de7\") "
Oct 31 22:20:10.880972 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.880928    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/869ed893-352c-4f8f-a174-b42afdeb4de7-home\" (UniqueName: \"kubernetes.io/empty-dir/869ed893-352c-4f8f-a174-b42afdeb4de7-home\") pod \"869ed893-352c-4f8f-a174-b42afdeb4de7\" (UID: \"869ed893-352c-4f8f-a174-b42afdeb4de7\") "
Oct 31 22:20:10.881052 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.880993    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/869ed893-352c-4f8f-a174-b42afdeb4de7-kube-api-access-v4lf7\" (UniqueName: \"kubernetes.io/projected/869ed893-352c-4f8f-a174-b42afdeb4de7-kube-api-access-v4lf7\") pod \"869ed893-352c-4f8f-a174-b42afdeb4de7\" (UID: \"869ed893-352c-4f8f-a174-b42afdeb4de7\") "
Oct 31 22:20:10.881052 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.881033    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/869ed893-352c-4f8f-a174-b42afdeb4de7-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/869ed893-352c-4f8f-a174-b42afdeb4de7-tmp-dir\") pod \"869ed893-352c-4f8f-a174-b42afdeb4de7\" (UID: \"869ed893-352c-4f8f-a174-b42afdeb4de7\") "
Oct 31 22:20:10.881162 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.881075    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/869ed893-352c-4f8f-a174-b42afdeb4de7-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/869ed893-352c-4f8f-a174-b42afdeb4de7-model-cache\") pod \"869ed893-352c-4f8f-a174-b42afdeb4de7\" (UID: \"869ed893-352c-4f8f-a174-b42afdeb4de7\") "
Oct 31 22:20:10.881223 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.881190    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/869ed893-352c-4f8f-a174-b42afdeb4de7-home" pod "869ed893-352c-4f8f-a174-b42afdeb4de7" (UID: "869ed893-352c-4f8f-a174-b42afdeb4de7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:20:10.881450 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.881426    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/869ed893-352c-4f8f-a174-b42afdeb4de7-model-cache" pod "869ed893-352c-4f8f-a174-b42afdeb4de7" (UID: "869ed893-352c-4f8f-a174-b42afdeb4de7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:20:10.881565 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.881446    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/869ed893-352c-4f8f-a174-b42afdeb4de7-tmp-dir" pod "869ed893-352c-4f8f-a174-b42afdeb4de7" (UID: "869ed893-352c-4f8f-a174-b42afdeb4de7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:20:10.881565 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.881487    4460 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/869ed893-352c-4f8f-a174-b42afdeb4de7-home\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:10.883423 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.883393    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/869ed893-352c-4f8f-a174-b42afdeb4de7-tls-certs" pod "869ed893-352c-4f8f-a174-b42afdeb4de7" (UID: "869ed893-352c-4f8f-a174-b42afdeb4de7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 22:20:10.883688 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.883661    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/869ed893-352c-4f8f-a174-b42afdeb4de7-dshm" pod "869ed893-352c-4f8f-a174-b42afdeb4de7" (UID: "869ed893-352c-4f8f-a174-b42afdeb4de7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:20:10.883781 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.883752    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/869ed893-352c-4f8f-a174-b42afdeb4de7-kube-api-access-v4lf7" pod "869ed893-352c-4f8f-a174-b42afdeb4de7" (UID: "869ed893-352c-4f8f-a174-b42afdeb4de7"). InnerVolumeSpecName "kube-api-access-v4lf7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:20:10.979083 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.979043    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:10.982069 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.982042    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-dshm\") pod \"d1078aa7-1947-45ce-b6bb-d6550ad61fcf\" (UID: \"d1078aa7-1947-45ce-b6bb-d6550ad61fcf\") "
Oct 31 22:20:10.982214 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.982082    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-model-cache\") pod \"d1078aa7-1947-45ce-b6bb-d6550ad61fcf\" (UID: \"d1078aa7-1947-45ce-b6bb-d6550ad61fcf\") "
Oct 31 22:20:10.982214 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.982100    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-home\" (UniqueName: \"kubernetes.io/empty-dir/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-home\") pod \"d1078aa7-1947-45ce-b6bb-d6550ad61fcf\" (UID: \"d1078aa7-1947-45ce-b6bb-d6550ad61fcf\") "
Oct 31 22:20:10.982214 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.982150    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-kube-api-access-fwzsf\" (UniqueName: \"kubernetes.io/projected/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-kube-api-access-fwzsf\") pod \"d1078aa7-1947-45ce-b6bb-d6550ad61fcf\" (UID: \"d1078aa7-1947-45ce-b6bb-d6550ad61fcf\") "
Oct 31 22:20:10.982214 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.982206    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-tmp-dir\") pod \"d1078aa7-1947-45ce-b6bb-d6550ad61fcf\" (UID: \"d1078aa7-1947-45ce-b6bb-d6550ad61fcf\") "
Oct 31 22:20:10.982461 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.982340    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-tls-certs\" (UniqueName: \"kubernetes.io/secret/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-tls-certs\") pod \"d1078aa7-1947-45ce-b6bb-d6550ad61fcf\" (UID: \"d1078aa7-1947-45ce-b6bb-d6550ad61fcf\") "
Oct 31 22:20:10.982461 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.982390    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-model-cache" pod "d1078aa7-1947-45ce-b6bb-d6550ad61fcf" (UID: "d1078aa7-1947-45ce-b6bb-d6550ad61fcf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:20:10.982461 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.982447    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-home" pod "d1078aa7-1947-45ce-b6bb-d6550ad61fcf" (UID: "d1078aa7-1947-45ce-b6bb-d6550ad61fcf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:20:10.982614 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.982463    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-tmp-dir" pod "d1078aa7-1947-45ce-b6bb-d6550ad61fcf" (UID: "d1078aa7-1947-45ce-b6bb-d6550ad61fcf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:20:10.982798 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.982778    4460 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-model-cache\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:10.982858 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.982799    4460 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-home\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:10.982858 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.982809    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4lf7\" (UniqueName: \"kubernetes.io/projected/869ed893-352c-4f8f-a174-b42afdeb4de7-kube-api-access-v4lf7\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:10.982858 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.982839    4460 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/869ed893-352c-4f8f-a174-b42afdeb4de7-tmp-dir\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:10.982858 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.982852    4460 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/869ed893-352c-4f8f-a174-b42afdeb4de7-model-cache\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:10.982858 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.982860    4460 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/869ed893-352c-4f8f-a174-b42afdeb4de7-tls-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:10.983168 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.982867    4460 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-tmp-dir\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:10.983168 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.982876    4460 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/869ed893-352c-4f8f-a174-b42afdeb4de7-dshm\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:10.984307 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.984280    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-dshm" pod "d1078aa7-1947-45ce-b6bb-d6550ad61fcf" (UID: "d1078aa7-1947-45ce-b6bb-d6550ad61fcf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:20:10.984423 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.984283    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-kube-api-access-fwzsf" pod "d1078aa7-1947-45ce-b6bb-d6550ad61fcf" (UID: "d1078aa7-1947-45ce-b6bb-d6550ad61fcf"). InnerVolumeSpecName "kube-api-access-fwzsf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:20:10.984465 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:10.984437    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-tls-certs" pod "d1078aa7-1947-45ce-b6bb-d6550ad61fcf" (UID: "d1078aa7-1947-45ce-b6bb-d6550ad61fcf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 22:20:11.084443 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:11.084376    4460 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-dshm\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:11.084443 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:11.084404    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwzsf\" (UniqueName: \"kubernetes.io/projected/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-kube-api-access-fwzsf\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:11.084443 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:11.084417    4460 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1078aa7-1947-45ce-b6bb-d6550ad61fcf-tls-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:11.410710 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:11.410617    4460 generic.go:356] "Generic (PLEG): container finished" podID="869ed893-352c-4f8f-a174-b42afdeb4de7" containerID="e0817b2f9f16f4aec1b829e0d1daf6425a927c079717e4a035f854e4b64b4b46" exitCode=0
Oct 31 22:20:11.410710 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:11.410658    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"
Oct 31 22:20:11.410710 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:11.410689    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh" event={"ID":"869ed893-352c-4f8f-a174-b42afdeb4de7","Type":"ContainerDied","Data":"e0817b2f9f16f4aec1b829e0d1daf6425a927c079717e4a035f854e4b64b4b46"}
Oct 31 22:20:11.411068 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:11.410735    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh" event={"ID":"869ed893-352c-4f8f-a174-b42afdeb4de7","Type":"ContainerDied","Data":"c13e9df596a4709caa644e1ca02b5810a028ca74112dd30a82c1e2dc4bf9ae53"}
Oct 31 22:20:11.411068 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:11.410756    4460 scope.go:122] "RemoveContainer" containerID="e0817b2f9f16f4aec1b829e0d1daf6425a927c079717e4a035f854e4b64b4b46"
Oct 31 22:20:11.412198 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:11.412171    4460 generic.go:356] "Generic (PLEG): container finished" podID="d1078aa7-1947-45ce-b6bb-d6550ad61fcf" containerID="af5b231520ddabdca0df6d92f164fe8c6fc8afa26d8ec336367e967e5f5d8350" exitCode=0
Oct 31 22:20:11.412312 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:11.412213    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"
Oct 31 22:20:11.412312 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:11.412255    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747" event={"ID":"d1078aa7-1947-45ce-b6bb-d6550ad61fcf","Type":"ContainerDied","Data":"af5b231520ddabdca0df6d92f164fe8c6fc8afa26d8ec336367e967e5f5d8350"}
Oct 31 22:20:11.412312 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:11.412296    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747" event={"ID":"d1078aa7-1947-45ce-b6bb-d6550ad61fcf","Type":"ContainerDied","Data":"bb20163a65ccda8ed79cbcffbb395a10dd64666b5ba1ef922fb8d9841017c158"}
Oct 31 22:20:11.421730 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:11.421711    4460 scope.go:122] "RemoveContainer" containerID="e0817b2f9f16f4aec1b829e0d1daf6425a927c079717e4a035f854e4b64b4b46"
Oct 31 22:20:11.422015 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:20:11.421990    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0817b2f9f16f4aec1b829e0d1daf6425a927c079717e4a035f854e4b64b4b46\": container with ID starting with e0817b2f9f16f4aec1b829e0d1daf6425a927c079717e4a035f854e4b64b4b46 not found: ID does not exist" containerID="e0817b2f9f16f4aec1b829e0d1daf6425a927c079717e4a035f854e4b64b4b46"
Oct 31 22:20:11.422083 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:11.422027    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0817b2f9f16f4aec1b829e0d1daf6425a927c079717e4a035f854e4b64b4b46"} err="failed to get container status \"e0817b2f9f16f4aec1b829e0d1daf6425a927c079717e4a035f854e4b64b4b46\": rpc error: code = NotFound desc = could not find container \"e0817b2f9f16f4aec1b829e0d1daf6425a927c079717e4a035f854e4b64b4b46\": container with ID starting with e0817b2f9f16f4aec1b829e0d1daf6425a927c079717e4a035f854e4b64b4b46 not found: ID does not exist"
Oct 31 22:20:11.422083 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:11.422049    4460 scope.go:122] "RemoveContainer" containerID="af5b231520ddabdca0df6d92f164fe8c6fc8afa26d8ec336367e967e5f5d8350"
Oct 31 22:20:11.431421 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:11.431401    4460 scope.go:122] "RemoveContainer" containerID="af5b231520ddabdca0df6d92f164fe8c6fc8afa26d8ec336367e967e5f5d8350"
Oct 31 22:20:11.431686 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:20:11.431665    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af5b231520ddabdca0df6d92f164fe8c6fc8afa26d8ec336367e967e5f5d8350\": container with ID starting with af5b231520ddabdca0df6d92f164fe8c6fc8afa26d8ec336367e967e5f5d8350 not found: ID does not exist" containerID="af5b231520ddabdca0df6d92f164fe8c6fc8afa26d8ec336367e967e5f5d8350"
Oct 31 22:20:11.431766 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:11.431691    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af5b231520ddabdca0df6d92f164fe8c6fc8afa26d8ec336367e967e5f5d8350"} err="failed to get container status \"af5b231520ddabdca0df6d92f164fe8c6fc8afa26d8ec336367e967e5f5d8350\": rpc error: code = NotFound desc = could not find container \"af5b231520ddabdca0df6d92f164fe8c6fc8afa26d8ec336367e967e5f5d8350\": container with ID starting with af5b231520ddabdca0df6d92f164fe8c6fc8afa26d8ec336367e967e5f5d8350 not found: ID does not exist"
Oct 31 22:20:11.467530 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:11.467506    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"]
Oct 31 22:20:11.480894 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:11.480874    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-994336/distinct-w1-994336-kserve-84c46f5f8-jf747"]
Oct 31 22:20:11.494489 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:11.494462    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:11.510678 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:11.510648    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"]
Oct 31 22:20:11.516740 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:11.516719    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-994336/distinct-2-w1-994336-kserve-d779f66fc-z5pgh"]
Oct 31 22:20:11.619638 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:11.619604    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:11.732962 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:11.732867    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:12.103697 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:12.103663    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:12.224483 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:12.224447    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="869ed893-352c-4f8f-a174-b42afdeb4de7" path="/var/lib/kubelet/pods/869ed893-352c-4f8f-a174-b42afdeb4de7/volumes"
Oct 31 22:20:12.224893 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:12.224879    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d1078aa7-1947-45ce-b6bb-d6550ad61fcf" path="/var/lib/kubelet/pods/d1078aa7-1947-45ce-b6bb-d6550ad61fcf/volumes"
Oct 31 22:20:13.157412 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:13.157377    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-b-56a6d4-6758676965-4l9dm" podUID="62d79c06-55b1-4b09-aa24-dd5519154921" containerName="payload-processing" containerID="cri-o://05daacd7275ef8dc3c747c0cfb4dce9f453f6b4612ea81360e181295d7ea2300" gracePeriod=25
Oct 31 22:20:13.192567 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:13.192514    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-56a6d4-7769777f8c-2jddx" podUID="4648d201-9e87-495c-b4aa-cb961fae6f26" containerName="payload-pre-processing" containerID="cri-o://8c80d3548b170139211b4a731e1face65c822d37f515e09ca4c07d0b945cbb3d" gracePeriod=25
Oct 31 22:20:13.422677 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:13.422596    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:14.576241 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:14.576203    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-b-56a6d4-6758676965-4l9dm" podUID="62d79c06-55b1-4b09-aa24-dd5519154921" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:20:14.576635 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:14.576203    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-56a6d4-7769777f8c-2jddx" podUID="4648d201-9e87-495c-b4aa-cb961fae6f26" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:20:14.758111 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:14.758072    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx"]
Oct 31 22:20:14.758381 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:14.758350    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx" podUID="aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac" containerName="maas-api" containerID="cri-o://6a1eadbfa4128a99de2c6750cf4a66f6626a55ce57d31b20cbe56d993ef7e1f5" gracePeriod=30
Oct 31 22:20:14.819432 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:14.819396    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:15.030480 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:15.030456    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx"
Oct 31 22:20:15.128514 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:15.128433    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac-openshift-service-ca\") pod \"aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac\" (UID: \"aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac\") "
Oct 31 22:20:15.128685 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:15.128586    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac-metrics-certs\" (UniqueName: \"kubernetes.io/secret/aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac-metrics-certs\") pod \"aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac\" (UID: \"aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac\") "
Oct 31 22:20:15.128685 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:15.128619    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac-kube-api-access-j98zs\" (UniqueName: \"kubernetes.io/projected/aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac-kube-api-access-j98zs\") pod \"aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac\" (UID: \"aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac\") "
Oct 31 22:20:15.128685 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:15.128678    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac-maas-api-tls\") pod \"aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac\" (UID: \"aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac\") "
Oct 31 22:20:15.128851 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:15.128752    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac-openshift-service-ca" pod "aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac" (UID: "aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:20:15.129060 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:15.129042    4460 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac-openshift-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:15.130738 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:15.130715    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac-metrics-certs" pod "aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac" (UID: "aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 22:20:15.130812 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:15.130744    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac-kube-api-access-j98zs" pod "aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac" (UID: "aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac"). InnerVolumeSpecName "kube-api-access-j98zs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:20:15.130812 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:15.130799    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac-maas-api-tls" pod "aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac" (UID: "aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 22:20:15.229860 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:15.229826    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac-maas-api-tls\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:15.229860 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:15.229852    4460 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac-metrics-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:15.229860 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:15.229864    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j98zs\" (UniqueName: \"kubernetes.io/projected/aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac-kube-api-access-j98zs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:15.287376 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:15.287339    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:15.436725 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:15.436636    4460 generic.go:356] "Generic (PLEG): container finished" podID="aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac" containerID="6a1eadbfa4128a99de2c6750cf4a66f6626a55ce57d31b20cbe56d993ef7e1f5" exitCode=0
Oct 31 22:20:15.436725 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:15.436686    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx"
Oct 31 22:20:15.436914 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:15.436727    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx" event={"ID":"aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac","Type":"ContainerDied","Data":"6a1eadbfa4128a99de2c6750cf4a66f6626a55ce57d31b20cbe56d993ef7e1f5"}
Oct 31 22:20:15.436914 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:15.436782    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx" event={"ID":"aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac","Type":"ContainerDied","Data":"a2b9a43a12b47e970f4457570ac1ec5353857db33341cf9d0e6923fe0050f44c"}
Oct 31 22:20:15.436914 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:15.436806    4460 scope.go:122] "RemoveContainer" containerID="6a1eadbfa4128a99de2c6750cf4a66f6626a55ce57d31b20cbe56d993ef7e1f5"
Oct 31 22:20:15.446473 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:15.446458    4460 scope.go:122] "RemoveContainer" containerID="6a1eadbfa4128a99de2c6750cf4a66f6626a55ce57d31b20cbe56d993ef7e1f5"
Oct 31 22:20:15.446783 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:20:15.446743    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a1eadbfa4128a99de2c6750cf4a66f6626a55ce57d31b20cbe56d993ef7e1f5\": container with ID starting with 6a1eadbfa4128a99de2c6750cf4a66f6626a55ce57d31b20cbe56d993ef7e1f5 not found: ID does not exist" containerID="6a1eadbfa4128a99de2c6750cf4a66f6626a55ce57d31b20cbe56d993ef7e1f5"
Oct 31 22:20:15.446877 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:15.446793    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a1eadbfa4128a99de2c6750cf4a66f6626a55ce57d31b20cbe56d993ef7e1f5"} err="failed to get container status \"6a1eadbfa4128a99de2c6750cf4a66f6626a55ce57d31b20cbe56d993ef7e1f5\": rpc error: code = NotFound desc = could not find container \"6a1eadbfa4128a99de2c6750cf4a66f6626a55ce57d31b20cbe56d993ef7e1f5\": container with ID starting with 6a1eadbfa4128a99de2c6750cf4a66f6626a55ce57d31b20cbe56d993ef7e1f5 not found: ID does not exist"
Oct 31 22:20:15.446982 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:15.446886    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:15.497270 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:15.497235    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx"]
Oct 31 22:20:15.500886 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:15.500859    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-994336-5df8d79df9-tdfdx"]
Oct 31 22:20:15.559008 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:15.558966    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:15.963210 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:15.963174    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 22:20:15.968178 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:15.968141    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:16.224423 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:16.224346    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac" path="/var/lib/kubelet/pods/aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac/volumes"
Oct 31 22:20:17.295951 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:17.295913    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:18.931318 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:18.931281    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:19.539492 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:19.539457    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:19.663750 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:19.663709    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:19.871050 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:19.870962    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:20.323304 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:20.323272    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:20.700035 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:20.699959    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:21.451637 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:20:21.451601    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:20:22.346723 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:22.346677    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:22.473082 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:22.469986    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:22.694875 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:22.694779    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:23.651768 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:23.651725    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 22:20:24.112105 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:24.112070    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:24.576880 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:24.576843    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-56a6d4-7769777f8c-2jddx" podUID="4648d201-9e87-495c-b4aa-cb961fae6f26" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:20:24.577081 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:24.576844    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-b-56a6d4-6758676965-4l9dm" podUID="62d79c06-55b1-4b09-aa24-dd5519154921" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:20:25.284301 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:25.284261    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"]
Oct 31 22:20:25.410704 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:25.410672    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:25.755572 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:25.755481    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:27.307575 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:27.307533    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:28.930052 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:28.930016    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:30.518775 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:30.518741    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:31.000820 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:31.000743    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-106555-849b8bdbd5-4trlx"]
Oct 31 22:20:31.037717 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:31.037685    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-106555-659c88c8f5-7hdjm"]
Oct 31 22:20:31.145109 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:31.145057    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:31.266444 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:31.266393    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:32.035030 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:32.034988    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:32.504747 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:32.504651    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:32.627230 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:32.627193    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:33.452078 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:33.452044    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-56a6d4-6758676965-4l9dm"
Oct 31 22:20:33.473079 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:33.473049    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-56a6d4-7769777f8c-2jddx"
Oct 31 22:20:33.523308 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:33.523283    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4648d201-9e87-495c-b4aa-cb961fae6f26-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4648d201-9e87-495c-b4aa-cb961fae6f26-plugins-config-volume\") pod \"4648d201-9e87-495c-b4aa-cb961fae6f26\" (UID: \"4648d201-9e87-495c-b4aa-cb961fae6f26\") "
Oct 31 22:20:33.523442 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:33.523329    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/62d79c06-55b1-4b09-aa24-dd5519154921-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62d79c06-55b1-4b09-aa24-dd5519154921-plugins-config-volume\") pod \"62d79c06-55b1-4b09-aa24-dd5519154921\" (UID: \"62d79c06-55b1-4b09-aa24-dd5519154921\") "
Oct 31 22:20:33.523442 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:33.523388    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/62d79c06-55b1-4b09-aa24-dd5519154921-kube-api-access-h7cwt\" (UniqueName: \"kubernetes.io/projected/62d79c06-55b1-4b09-aa24-dd5519154921-kube-api-access-h7cwt\") pod \"62d79c06-55b1-4b09-aa24-dd5519154921\" (UID: \"62d79c06-55b1-4b09-aa24-dd5519154921\") "
Oct 31 22:20:33.523536 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:33.523516    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4648d201-9e87-495c-b4aa-cb961fae6f26-kube-api-access-qjqq5\" (UniqueName: \"kubernetes.io/projected/4648d201-9e87-495c-b4aa-cb961fae6f26-kube-api-access-qjqq5\") pod \"4648d201-9e87-495c-b4aa-cb961fae6f26\" (UID: \"4648d201-9e87-495c-b4aa-cb961fae6f26\") "
Oct 31 22:20:33.523718 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:33.523698    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4648d201-9e87-495c-b4aa-cb961fae6f26-plugins-config-volume" pod "4648d201-9e87-495c-b4aa-cb961fae6f26" (UID: "4648d201-9e87-495c-b4aa-cb961fae6f26"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:20:33.523819 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:33.523789    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62d79c06-55b1-4b09-aa24-dd5519154921-plugins-config-volume" pod "62d79c06-55b1-4b09-aa24-dd5519154921" (UID: "62d79c06-55b1-4b09-aa24-dd5519154921"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:20:33.523930 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:33.523864    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4648d201-9e87-495c-b4aa-cb961fae6f26-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:33.523930 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:33.523878    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62d79c06-55b1-4b09-aa24-dd5519154921-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:33.525518 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:33.525492    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62d79c06-55b1-4b09-aa24-dd5519154921-kube-api-access-h7cwt" pod "62d79c06-55b1-4b09-aa24-dd5519154921" (UID: "62d79c06-55b1-4b09-aa24-dd5519154921"). InnerVolumeSpecName "kube-api-access-h7cwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:20:33.525610 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:33.525584    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4648d201-9e87-495c-b4aa-cb961fae6f26-kube-api-access-qjqq5" pod "4648d201-9e87-495c-b4aa-cb961fae6f26" (UID: "4648d201-9e87-495c-b4aa-cb961fae6f26"). InnerVolumeSpecName "kube-api-access-qjqq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:20:33.552286 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:33.552259    4460 generic.go:356] "Generic (PLEG): container finished" podID="62d79c06-55b1-4b09-aa24-dd5519154921" containerID="05daacd7275ef8dc3c747c0cfb4dce9f453f6b4612ea81360e181295d7ea2300" exitCode=0
Oct 31 22:20:33.552416 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:33.552303    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-56a6d4-6758676965-4l9dm"
Oct 31 22:20:33.552416 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:33.552341    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-56a6d4-6758676965-4l9dm" event={"ID":"62d79c06-55b1-4b09-aa24-dd5519154921","Type":"ContainerDied","Data":"05daacd7275ef8dc3c747c0cfb4dce9f453f6b4612ea81360e181295d7ea2300"}
Oct 31 22:20:33.552532 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:33.552422    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-56a6d4-6758676965-4l9dm" event={"ID":"62d79c06-55b1-4b09-aa24-dd5519154921","Type":"ContainerDied","Data":"f81c12bd1387fbf5cf27ac6703ee8ff28f4d169a2c40e572d2bab8ed4a0b143b"}
Oct 31 22:20:33.552532 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:33.552449    4460 scope.go:122] "RemoveContainer" containerID="05daacd7275ef8dc3c747c0cfb4dce9f453f6b4612ea81360e181295d7ea2300"
Oct 31 22:20:33.553955 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:33.553917    4460 generic.go:356] "Generic (PLEG): container finished" podID="4648d201-9e87-495c-b4aa-cb961fae6f26" containerID="8c80d3548b170139211b4a731e1face65c822d37f515e09ca4c07d0b945cbb3d" exitCode=0
Oct 31 22:20:33.554065 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:33.553978    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-56a6d4-7769777f8c-2jddx" event={"ID":"4648d201-9e87-495c-b4aa-cb961fae6f26","Type":"ContainerDied","Data":"8c80d3548b170139211b4a731e1face65c822d37f515e09ca4c07d0b945cbb3d"}
Oct 31 22:20:33.554065 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:33.553992    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-56a6d4-7769777f8c-2jddx"
Oct 31 22:20:33.554065 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:33.554042    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-56a6d4-7769777f8c-2jddx" event={"ID":"4648d201-9e87-495c-b4aa-cb961fae6f26","Type":"ContainerDied","Data":"270a878cce0e49bd5489c73cbaba24639a238c826b5576019b771cd494c08d9c"}
Oct 31 22:20:33.563294 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:33.563278    4460 scope.go:122] "RemoveContainer" containerID="05daacd7275ef8dc3c747c0cfb4dce9f453f6b4612ea81360e181295d7ea2300"
Oct 31 22:20:33.563612 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:20:33.563587    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05daacd7275ef8dc3c747c0cfb4dce9f453f6b4612ea81360e181295d7ea2300\": container with ID starting with 05daacd7275ef8dc3c747c0cfb4dce9f453f6b4612ea81360e181295d7ea2300 not found: ID does not exist" containerID="05daacd7275ef8dc3c747c0cfb4dce9f453f6b4612ea81360e181295d7ea2300"
Oct 31 22:20:33.563699 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:33.563623    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05daacd7275ef8dc3c747c0cfb4dce9f453f6b4612ea81360e181295d7ea2300"} err="failed to get container status \"05daacd7275ef8dc3c747c0cfb4dce9f453f6b4612ea81360e181295d7ea2300\": rpc error: code = NotFound desc = could not find container \"05daacd7275ef8dc3c747c0cfb4dce9f453f6b4612ea81360e181295d7ea2300\": container with ID starting with 05daacd7275ef8dc3c747c0cfb4dce9f453f6b4612ea81360e181295d7ea2300 not found: ID does not exist"
Oct 31 22:20:33.563699 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:33.563642    4460 scope.go:122] "RemoveContainer" containerID="8c80d3548b170139211b4a731e1face65c822d37f515e09ca4c07d0b945cbb3d"
Oct 31 22:20:33.572342 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:33.572325    4460 scope.go:122] "RemoveContainer" containerID="8c80d3548b170139211b4a731e1face65c822d37f515e09ca4c07d0b945cbb3d"
Oct 31 22:20:33.572631 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:20:33.572612    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c80d3548b170139211b4a731e1face65c822d37f515e09ca4c07d0b945cbb3d\": container with ID starting with 8c80d3548b170139211b4a731e1face65c822d37f515e09ca4c07d0b945cbb3d not found: ID does not exist" containerID="8c80d3548b170139211b4a731e1face65c822d37f515e09ca4c07d0b945cbb3d"
Oct 31 22:20:33.572683 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:33.572639    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c80d3548b170139211b4a731e1face65c822d37f515e09ca4c07d0b945cbb3d"} err="failed to get container status \"8c80d3548b170139211b4a731e1face65c822d37f515e09ca4c07d0b945cbb3d\": rpc error: code = NotFound desc = could not find container \"8c80d3548b170139211b4a731e1face65c822d37f515e09ca4c07d0b945cbb3d\": container with ID starting with 8c80d3548b170139211b4a731e1face65c822d37f515e09ca4c07d0b945cbb3d not found: ID does not exist"
Oct 31 22:20:33.608288 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:33.608266    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-56a6d4-7769777f8c-2jddx"]
Oct 31 22:20:33.613421 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:33.613401    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-56a6d4-7769777f8c-2jddx"]
Oct 31 22:20:33.625040 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:33.625011    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7cwt\" (UniqueName: \"kubernetes.io/projected/62d79c06-55b1-4b09-aa24-dd5519154921-kube-api-access-h7cwt\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:33.625122 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:33.625040    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjqq5\" (UniqueName: \"kubernetes.io/projected/4648d201-9e87-495c-b4aa-cb961fae6f26-kube-api-access-qjqq5\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:33.650392 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:33.650365    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-56a6d4-6758676965-4l9dm"]
Oct 31 22:20:33.654197 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:33.654173    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-56a6d4-6758676965-4l9dm"]
Oct 31 22:20:34.070649 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:34.070614    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:34.224352 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:34.224318    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4648d201-9e87-495c-b4aa-cb961fae6f26" path="/var/lib/kubelet/pods/4648d201-9e87-495c-b4aa-cb961fae6f26/volumes"
Oct 31 22:20:34.224775 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:34.224752    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="62d79c06-55b1-4b09-aa24-dd5519154921" path="/var/lib/kubelet/pods/62d79c06-55b1-4b09-aa24-dd5519154921/volumes"
Oct 31 22:20:36.001512 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:36.001468    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w3-106555-849b8bdbd5-4trlx" podUID="342810a1-1edf-4dde-ac1d-b8844e81d5ac" containerName="payload-processing" containerID="cri-o://39c1609bab709638d5b200c461eeeaf52d3c142cb7c0b35e8b8ed4de76847b38" gracePeriod=25
Oct 31 22:20:36.038182 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:36.038144    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-106555-659c88c8f5-7hdjm" podUID="9fb4edcd-09a0-4f56-9391-574b046c514e" containerName="payload-pre-processing" containerID="cri-o://1c841413ed6dc40751b20a30e77ce57fe9da641439459fcedb8b39acbe5a3b11" gracePeriod=25
Oct 31 22:20:36.268146 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:36.268107    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-106555-659c88c8f5-7hdjm" podUID="9fb4edcd-09a0-4f56-9391-574b046c514e" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:20:36.268330 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:36.268174    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-106555-849b8bdbd5-4trlx" podUID="342810a1-1edf-4dde-ac1d-b8844e81d5ac" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:20:39.994402 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:39.994362    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:40.304606 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:40.304565    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt" podUID="9771611f-879a-44f6-9c1a-f8647dc2d841" containerName="main" containerID="cri-o://73d9aaf9d7c0c2fe045ecf2b1a443e5efdee1811083a06bc560d6c8256490b9c" gracePeriod=45
Oct 31 22:20:40.553340 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:40.553302    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 22:20:40.593411 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:40.593331    4460 generic.go:356] "Generic (PLEG): container finished" podID="9771611f-879a-44f6-9c1a-f8647dc2d841" containerID="73d9aaf9d7c0c2fe045ecf2b1a443e5efdee1811083a06bc560d6c8256490b9c" exitCode=0
Oct 31 22:20:40.593411 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:40.593382    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"
Oct 31 22:20:40.593620 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:40.593383    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt" event={"ID":"9771611f-879a-44f6-9c1a-f8647dc2d841","Type":"ContainerDied","Data":"73d9aaf9d7c0c2fe045ecf2b1a443e5efdee1811083a06bc560d6c8256490b9c"}
Oct 31 22:20:40.593620 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:40.593514    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt" event={"ID":"9771611f-879a-44f6-9c1a-f8647dc2d841","Type":"ContainerDied","Data":"187ea0133245631e2eba9644ff0c9cb82636271953b240038587b85968fa472e"}
Oct 31 22:20:40.593620 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:40.593540    4460 scope.go:122] "RemoveContainer" containerID="73d9aaf9d7c0c2fe045ecf2b1a443e5efdee1811083a06bc560d6c8256490b9c"
Oct 31 22:20:40.605351 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:40.605253    4460 scope.go:122] "RemoveContainer" containerID="73d9aaf9d7c0c2fe045ecf2b1a443e5efdee1811083a06bc560d6c8256490b9c"
Oct 31 22:20:40.605605 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:20:40.605582    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73d9aaf9d7c0c2fe045ecf2b1a443e5efdee1811083a06bc560d6c8256490b9c\": container with ID starting with 73d9aaf9d7c0c2fe045ecf2b1a443e5efdee1811083a06bc560d6c8256490b9c not found: ID does not exist" containerID="73d9aaf9d7c0c2fe045ecf2b1a443e5efdee1811083a06bc560d6c8256490b9c"
Oct 31 22:20:40.605707 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:40.605616    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73d9aaf9d7c0c2fe045ecf2b1a443e5efdee1811083a06bc560d6c8256490b9c"} err="failed to get container status \"73d9aaf9d7c0c2fe045ecf2b1a443e5efdee1811083a06bc560d6c8256490b9c\": rpc error: code = NotFound desc = could not find container \"73d9aaf9d7c0c2fe045ecf2b1a443e5efdee1811083a06bc560d6c8256490b9c\": container with ID starting with 73d9aaf9d7c0c2fe045ecf2b1a443e5efdee1811083a06bc560d6c8256490b9c not found: ID does not exist"
Oct 31 22:20:40.687445 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:40.687413    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9771611f-879a-44f6-9c1a-f8647dc2d841-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9771611f-879a-44f6-9c1a-f8647dc2d841-model-cache\") pod \"9771611f-879a-44f6-9c1a-f8647dc2d841\" (UID: \"9771611f-879a-44f6-9c1a-f8647dc2d841\") "
Oct 31 22:20:40.687640 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:40.687452    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9771611f-879a-44f6-9c1a-f8647dc2d841-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9771611f-879a-44f6-9c1a-f8647dc2d841-tmp-dir\") pod \"9771611f-879a-44f6-9c1a-f8647dc2d841\" (UID: \"9771611f-879a-44f6-9c1a-f8647dc2d841\") "
Oct 31 22:20:40.687640 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:40.687493    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9771611f-879a-44f6-9c1a-f8647dc2d841-tls-certs\" (UniqueName: \"kubernetes.io/secret/9771611f-879a-44f6-9c1a-f8647dc2d841-tls-certs\") pod \"9771611f-879a-44f6-9c1a-f8647dc2d841\" (UID: \"9771611f-879a-44f6-9c1a-f8647dc2d841\") "
Oct 31 22:20:40.687640 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:40.687529    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9771611f-879a-44f6-9c1a-f8647dc2d841-home\" (UniqueName: \"kubernetes.io/empty-dir/9771611f-879a-44f6-9c1a-f8647dc2d841-home\") pod \"9771611f-879a-44f6-9c1a-f8647dc2d841\" (UID: \"9771611f-879a-44f6-9c1a-f8647dc2d841\") "
Oct 31 22:20:40.687640 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:40.687585    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9771611f-879a-44f6-9c1a-f8647dc2d841-dshm\" (UniqueName: \"kubernetes.io/empty-dir/9771611f-879a-44f6-9c1a-f8647dc2d841-dshm\") pod \"9771611f-879a-44f6-9c1a-f8647dc2d841\" (UID: \"9771611f-879a-44f6-9c1a-f8647dc2d841\") "
Oct 31 22:20:40.687877 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:40.687696    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9771611f-879a-44f6-9c1a-f8647dc2d841-kube-api-access-vpnfb\" (UniqueName: \"kubernetes.io/projected/9771611f-879a-44f6-9c1a-f8647dc2d841-kube-api-access-vpnfb\") pod \"9771611f-879a-44f6-9c1a-f8647dc2d841\" (UID: \"9771611f-879a-44f6-9c1a-f8647dc2d841\") "
Oct 31 22:20:40.687877 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:40.687731    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9771611f-879a-44f6-9c1a-f8647dc2d841-model-cache" pod "9771611f-879a-44f6-9c1a-f8647dc2d841" (UID: "9771611f-879a-44f6-9c1a-f8647dc2d841"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:20:40.687877 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:40.687794    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9771611f-879a-44f6-9c1a-f8647dc2d841-tmp-dir" pod "9771611f-879a-44f6-9c1a-f8647dc2d841" (UID: "9771611f-879a-44f6-9c1a-f8647dc2d841"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:20:40.688110 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:40.688089    4460 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9771611f-879a-44f6-9c1a-f8647dc2d841-model-cache\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:40.688182 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:40.688115    4460 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9771611f-879a-44f6-9c1a-f8647dc2d841-tmp-dir\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:40.688323 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:40.688301    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9771611f-879a-44f6-9c1a-f8647dc2d841-home" pod "9771611f-879a-44f6-9c1a-f8647dc2d841" (UID: "9771611f-879a-44f6-9c1a-f8647dc2d841"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:20:40.690296 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:40.690272    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9771611f-879a-44f6-9c1a-f8647dc2d841-tls-certs" pod "9771611f-879a-44f6-9c1a-f8647dc2d841" (UID: "9771611f-879a-44f6-9c1a-f8647dc2d841"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 22:20:40.690448 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:40.690361    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9771611f-879a-44f6-9c1a-f8647dc2d841-kube-api-access-vpnfb" pod "9771611f-879a-44f6-9c1a-f8647dc2d841" (UID: "9771611f-879a-44f6-9c1a-f8647dc2d841"). InnerVolumeSpecName "kube-api-access-vpnfb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:20:40.690448 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:40.690392    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9771611f-879a-44f6-9c1a-f8647dc2d841-dshm" pod "9771611f-879a-44f6-9c1a-f8647dc2d841" (UID: "9771611f-879a-44f6-9c1a-f8647dc2d841"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:20:40.789249 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:40.789216    4460 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9771611f-879a-44f6-9c1a-f8647dc2d841-dshm\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:40.789249 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:40.789248    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vpnfb\" (UniqueName: \"kubernetes.io/projected/9771611f-879a-44f6-9c1a-f8647dc2d841-kube-api-access-vpnfb\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:40.789453 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:40.789258    4460 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9771611f-879a-44f6-9c1a-f8647dc2d841-tls-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:40.789453 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:40.789267    4460 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9771611f-879a-44f6-9c1a-f8647dc2d841-home\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:40.941488 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:40.941452    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"]
Oct 31 22:20:40.955076 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:40.955045    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-106555/distinct-2-w3-106555-kserve-78f45b6fc5-m44xt"]
Oct 31 22:20:42.224282 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:42.224249    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9771611f-879a-44f6-9c1a-f8647dc2d841" path="/var/lib/kubelet/pods/9771611f-879a-44f6-9c1a-f8647dc2d841/volumes"
Oct 31 22:20:46.267887 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:46.267846    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-106555-849b8bdbd5-4trlx" podUID="342810a1-1edf-4dde-ac1d-b8844e81d5ac" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:20:46.268420 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:46.267858    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-106555-659c88c8f5-7hdjm" podUID="9fb4edcd-09a0-4f56-9391-574b046c514e" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 22:20:47.049560 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:47.049519    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 22:20:48.526568 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:48.526534    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn"]
Oct 31 22:20:48.527022 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:48.526758    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn" podUID="8ee0ad1f-c884-4875-8af0-b494cf74c092" containerName="maas-api" containerID="cri-o://0b6bd0423aa1d0e19c220da129343ef6b2a71b4c2646631230ddcff22680b950" gracePeriod=30
Oct 31 22:20:48.592729 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:48.592699    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:48.784342 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:48.784278    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn"
Oct 31 22:20:48.971842 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:48.971805    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8ee0ad1f-c884-4875-8af0-b494cf74c092-kube-api-access-9pjk5\" (UniqueName: \"kubernetes.io/projected/8ee0ad1f-c884-4875-8af0-b494cf74c092-kube-api-access-9pjk5\") pod \"8ee0ad1f-c884-4875-8af0-b494cf74c092\" (UID: \"8ee0ad1f-c884-4875-8af0-b494cf74c092\") "
Oct 31 22:20:48.972035 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:48.971851    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8ee0ad1f-c884-4875-8af0-b494cf74c092-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ee0ad1f-c884-4875-8af0-b494cf74c092-openshift-service-ca\") pod \"8ee0ad1f-c884-4875-8af0-b494cf74c092\" (UID: \"8ee0ad1f-c884-4875-8af0-b494cf74c092\") "
Oct 31 22:20:48.972035 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:48.971890    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8ee0ad1f-c884-4875-8af0-b494cf74c092-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ee0ad1f-c884-4875-8af0-b494cf74c092-maas-api-tls\") pod \"8ee0ad1f-c884-4875-8af0-b494cf74c092\" (UID: \"8ee0ad1f-c884-4875-8af0-b494cf74c092\") "
Oct 31 22:20:48.972143 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:48.972067    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8ee0ad1f-c884-4875-8af0-b494cf74c092-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ee0ad1f-c884-4875-8af0-b494cf74c092-metrics-certs\") pod \"8ee0ad1f-c884-4875-8af0-b494cf74c092\" (UID: \"8ee0ad1f-c884-4875-8af0-b494cf74c092\") "
Oct 31 22:20:48.972354 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:48.972317    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ee0ad1f-c884-4875-8af0-b494cf74c092-openshift-service-ca" pod "8ee0ad1f-c884-4875-8af0-b494cf74c092" (UID: "8ee0ad1f-c884-4875-8af0-b494cf74c092"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:20:48.972458 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:48.972434    4460 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ee0ad1f-c884-4875-8af0-b494cf74c092-openshift-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:48.974047 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:48.974011    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ee0ad1f-c884-4875-8af0-b494cf74c092-kube-api-access-9pjk5" pod "8ee0ad1f-c884-4875-8af0-b494cf74c092" (UID: "8ee0ad1f-c884-4875-8af0-b494cf74c092"). InnerVolumeSpecName "kube-api-access-9pjk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:20:48.974158 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:48.974057    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ee0ad1f-c884-4875-8af0-b494cf74c092-maas-api-tls" pod "8ee0ad1f-c884-4875-8af0-b494cf74c092" (UID: "8ee0ad1f-c884-4875-8af0-b494cf74c092"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 22:20:48.974203 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:48.974171    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ee0ad1f-c884-4875-8af0-b494cf74c092-metrics-certs" pod "8ee0ad1f-c884-4875-8af0-b494cf74c092" (UID: "8ee0ad1f-c884-4875-8af0-b494cf74c092"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 22:20:49.032523 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:49.032486    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:49.072983 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:49.072908    4460 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ee0ad1f-c884-4875-8af0-b494cf74c092-metrics-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:49.072983 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:49.072935    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9pjk5\" (UniqueName: \"kubernetes.io/projected/8ee0ad1f-c884-4875-8af0-b494cf74c092-kube-api-access-9pjk5\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:49.072983 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:49.072967    4460 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ee0ad1f-c884-4875-8af0-b494cf74c092-maas-api-tls\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:49.157001 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:49.156969    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:49.632756 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:49.632723    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:49.638658 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:49.638629    4460 generic.go:356] "Generic (PLEG): container finished" podID="8ee0ad1f-c884-4875-8af0-b494cf74c092" containerID="0b6bd0423aa1d0e19c220da129343ef6b2a71b4c2646631230ddcff22680b950" exitCode=0
Oct 31 22:20:49.638786 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:49.638677    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn"
Oct 31 22:20:49.638786 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:49.638676    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn" event={"ID":"8ee0ad1f-c884-4875-8af0-b494cf74c092","Type":"ContainerDied","Data":"0b6bd0423aa1d0e19c220da129343ef6b2a71b4c2646631230ddcff22680b950"}
Oct 31 22:20:49.638861 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:49.638794    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn" event={"ID":"8ee0ad1f-c884-4875-8af0-b494cf74c092","Type":"ContainerDied","Data":"8de53173304a411e97c2083b85ecda8a715f7e5c812bad671cc614853c8e7434"}
Oct 31 22:20:49.638861 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:49.638817    4460 scope.go:122] "RemoveContainer" containerID="0b6bd0423aa1d0e19c220da129343ef6b2a71b4c2646631230ddcff22680b950"
Oct 31 22:20:49.648129 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:49.648110    4460 scope.go:122] "RemoveContainer" containerID="0b6bd0423aa1d0e19c220da129343ef6b2a71b4c2646631230ddcff22680b950"
Oct 31 22:20:49.648411 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:20:49.648391    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b6bd0423aa1d0e19c220da129343ef6b2a71b4c2646631230ddcff22680b950\": container with ID starting with 0b6bd0423aa1d0e19c220da129343ef6b2a71b4c2646631230ddcff22680b950 not found: ID does not exist" containerID="0b6bd0423aa1d0e19c220da129343ef6b2a71b4c2646631230ddcff22680b950"
Oct 31 22:20:49.648474 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:49.648418    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b6bd0423aa1d0e19c220da129343ef6b2a71b4c2646631230ddcff22680b950"} err="failed to get container status \"0b6bd0423aa1d0e19c220da129343ef6b2a71b4c2646631230ddcff22680b950\": rpc error: code = NotFound desc = could not find container \"0b6bd0423aa1d0e19c220da129343ef6b2a71b4c2646631230ddcff22680b950\": container with ID starting with 0b6bd0423aa1d0e19c220da129343ef6b2a71b4c2646631230ddcff22680b950 not found: ID does not exist"
Oct 31 22:20:49.697632 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:49.697610    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn"]
Oct 31 22:20:49.711367 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:49.711344    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-106555-55bc7f47d5-r9jhn"]
Oct 31 22:20:50.229138 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:50.229102    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8ee0ad1f-c884-4875-8af0-b494cf74c092" path="/var/lib/kubelet/pods/8ee0ad1f-c884-4875-8af0-b494cf74c092/volumes"
Oct 31 22:20:51.451260 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:20:51.451218    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:20:51.660318 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:51.660286    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:54.338277 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:54.338246    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:54.450518 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:54.450476    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:55.363053 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:55.363019    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-8p5kz"]
Oct 31 22:20:56.114691 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.114661    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:56.223619 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.223542    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:56.280012 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.279988    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-106555-849b8bdbd5-4trlx"
Oct 31 22:20:56.314906 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.314881    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-106555-659c88c8f5-7hdjm"
Oct 31 22:20:56.337271 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.337241    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/342810a1-1edf-4dde-ac1d-b8844e81d5ac-kube-api-access-84jrn\" (UniqueName: \"kubernetes.io/projected/342810a1-1edf-4dde-ac1d-b8844e81d5ac-kube-api-access-84jrn\") pod \"342810a1-1edf-4dde-ac1d-b8844e81d5ac\" (UID: \"342810a1-1edf-4dde-ac1d-b8844e81d5ac\") "
Oct 31 22:20:56.337416 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.337322    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9fb4edcd-09a0-4f56-9391-574b046c514e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fb4edcd-09a0-4f56-9391-574b046c514e-plugins-config-volume\") pod \"9fb4edcd-09a0-4f56-9391-574b046c514e\" (UID: \"9fb4edcd-09a0-4f56-9391-574b046c514e\") "
Oct 31 22:20:56.337416 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.337407    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9fb4edcd-09a0-4f56-9391-574b046c514e-kube-api-access-v58mz\" (UniqueName: \"kubernetes.io/projected/9fb4edcd-09a0-4f56-9391-574b046c514e-kube-api-access-v58mz\") pod \"9fb4edcd-09a0-4f56-9391-574b046c514e\" (UID: \"9fb4edcd-09a0-4f56-9391-574b046c514e\") "
Oct 31 22:20:56.337539 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.337466    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/342810a1-1edf-4dde-ac1d-b8844e81d5ac-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/342810a1-1edf-4dde-ac1d-b8844e81d5ac-plugins-config-volume\") pod \"342810a1-1edf-4dde-ac1d-b8844e81d5ac\" (UID: \"342810a1-1edf-4dde-ac1d-b8844e81d5ac\") "
Oct 31 22:20:56.337832 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.337805    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fb4edcd-09a0-4f56-9391-574b046c514e-plugins-config-volume" pod "9fb4edcd-09a0-4f56-9391-574b046c514e" (UID: "9fb4edcd-09a0-4f56-9391-574b046c514e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:20:56.337969 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.337847    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/342810a1-1edf-4dde-ac1d-b8844e81d5ac-plugins-config-volume" pod "342810a1-1edf-4dde-ac1d-b8844e81d5ac" (UID: "342810a1-1edf-4dde-ac1d-b8844e81d5ac"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:20:56.339472 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.339447    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fb4edcd-09a0-4f56-9391-574b046c514e-kube-api-access-v58mz" pod "9fb4edcd-09a0-4f56-9391-574b046c514e" (UID: "9fb4edcd-09a0-4f56-9391-574b046c514e"). InnerVolumeSpecName "kube-api-access-v58mz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:20:56.339573 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.339483    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/342810a1-1edf-4dde-ac1d-b8844e81d5ac-kube-api-access-84jrn" pod "342810a1-1edf-4dde-ac1d-b8844e81d5ac" (UID: "342810a1-1edf-4dde-ac1d-b8844e81d5ac"). InnerVolumeSpecName "kube-api-access-84jrn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:20:56.423950 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.423892    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"]
Oct 31 22:20:56.424800 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.424760    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z" podUID="ea04c855-12eb-4628-a947-3b4e8321bef1" containerName="istio-proxy" containerID="cri-o://b816bf29bf7a227b1ab47d8eae86b3be3f257d6564a18b8afc3556945da1757f" gracePeriod=30
Oct 31 22:20:56.439263 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.439234    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v58mz\" (UniqueName: \"kubernetes.io/projected/9fb4edcd-09a0-4f56-9391-574b046c514e-kube-api-access-v58mz\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:56.439263 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.439262    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/342810a1-1edf-4dde-ac1d-b8844e81d5ac-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:56.439409 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.439271    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84jrn\" (UniqueName: \"kubernetes.io/projected/342810a1-1edf-4dde-ac1d-b8844e81d5ac-kube-api-access-84jrn\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:56.439409 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.439280    4460 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fb4edcd-09a0-4f56-9391-574b046c514e-plugins-config-volume\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:20:56.449088 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.449060    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:20:56.675280 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.675244    4460 generic.go:356] "Generic (PLEG): container finished" podID="9fb4edcd-09a0-4f56-9391-574b046c514e" containerID="1c841413ed6dc40751b20a30e77ce57fe9da641439459fcedb8b39acbe5a3b11" exitCode=0
Oct 31 22:20:56.675475 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.675307    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-106555-659c88c8f5-7hdjm"
Oct 31 22:20:56.675475 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.675328    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-106555-659c88c8f5-7hdjm" event={"ID":"9fb4edcd-09a0-4f56-9391-574b046c514e","Type":"ContainerDied","Data":"1c841413ed6dc40751b20a30e77ce57fe9da641439459fcedb8b39acbe5a3b11"}
Oct 31 22:20:56.675475 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.675384    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-106555-659c88c8f5-7hdjm" event={"ID":"9fb4edcd-09a0-4f56-9391-574b046c514e","Type":"ContainerDied","Data":"13b73916edaa4ba98555b4a9a67e8b7929655fbf73d70d7b1c8e8c4b9ef597d1"}
Oct 31 22:20:56.675475 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.675413    4460 scope.go:122] "RemoveContainer" containerID="1c841413ed6dc40751b20a30e77ce57fe9da641439459fcedb8b39acbe5a3b11"
Oct 31 22:20:56.676666 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.676634    4460 generic.go:356] "Generic (PLEG): container finished" podID="342810a1-1edf-4dde-ac1d-b8844e81d5ac" containerID="39c1609bab709638d5b200c461eeeaf52d3c142cb7c0b35e8b8ed4de76847b38" exitCode=0
Oct 31 22:20:56.676733 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.676687    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-106555-849b8bdbd5-4trlx" event={"ID":"342810a1-1edf-4dde-ac1d-b8844e81d5ac","Type":"ContainerDied","Data":"39c1609bab709638d5b200c461eeeaf52d3c142cb7c0b35e8b8ed4de76847b38"}
Oct 31 22:20:56.676733 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.676693    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-106555-849b8bdbd5-4trlx"
Oct 31 22:20:56.676733 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.676718    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-106555-849b8bdbd5-4trlx" event={"ID":"342810a1-1edf-4dde-ac1d-b8844e81d5ac","Type":"ContainerDied","Data":"605b4528c0d48953915555c851b8fda6425fb5fffbd4cb35341c30ada4f6de06"}
Oct 31 22:20:56.684324 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.684303    4460 scope.go:122] "RemoveContainer" containerID="1c841413ed6dc40751b20a30e77ce57fe9da641439459fcedb8b39acbe5a3b11"
Oct 31 22:20:56.684613 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:20:56.684588    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c841413ed6dc40751b20a30e77ce57fe9da641439459fcedb8b39acbe5a3b11\": container with ID starting with 1c841413ed6dc40751b20a30e77ce57fe9da641439459fcedb8b39acbe5a3b11 not found: ID does not exist" containerID="1c841413ed6dc40751b20a30e77ce57fe9da641439459fcedb8b39acbe5a3b11"
Oct 31 22:20:56.684675 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.684624    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c841413ed6dc40751b20a30e77ce57fe9da641439459fcedb8b39acbe5a3b11"} err="failed to get container status \"1c841413ed6dc40751b20a30e77ce57fe9da641439459fcedb8b39acbe5a3b11\": rpc error: code = NotFound desc = could not find container \"1c841413ed6dc40751b20a30e77ce57fe9da641439459fcedb8b39acbe5a3b11\": container with ID starting with 1c841413ed6dc40751b20a30e77ce57fe9da641439459fcedb8b39acbe5a3b11 not found: ID does not exist"
Oct 31 22:20:56.684675 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.684648    4460 scope.go:122] "RemoveContainer" containerID="39c1609bab709638d5b200c461eeeaf52d3c142cb7c0b35e8b8ed4de76847b38"
Oct 31 22:20:56.692844 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.692824    4460 scope.go:122] "RemoveContainer" containerID="39c1609bab709638d5b200c461eeeaf52d3c142cb7c0b35e8b8ed4de76847b38"
Oct 31 22:20:56.693111 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:20:56.693092    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39c1609bab709638d5b200c461eeeaf52d3c142cb7c0b35e8b8ed4de76847b38\": container with ID starting with 39c1609bab709638d5b200c461eeeaf52d3c142cb7c0b35e8b8ed4de76847b38 not found: ID does not exist" containerID="39c1609bab709638d5b200c461eeeaf52d3c142cb7c0b35e8b8ed4de76847b38"
Oct 31 22:20:56.693166 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.693120    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39c1609bab709638d5b200c461eeeaf52d3c142cb7c0b35e8b8ed4de76847b38"} err="failed to get container status \"39c1609bab709638d5b200c461eeeaf52d3c142cb7c0b35e8b8ed4de76847b38\": rpc error: code = NotFound desc = could not find container \"39c1609bab709638d5b200c461eeeaf52d3c142cb7c0b35e8b8ed4de76847b38\": container with ID starting with 39c1609bab709638d5b200c461eeeaf52d3c142cb7c0b35e8b8ed4de76847b38 not found: ID does not exist"
Oct 31 22:20:56.730122 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.730092    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-106555-659c88c8f5-7hdjm"]
Oct 31 22:20:56.734490 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.734464    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-106555-659c88c8f5-7hdjm"]
Oct 31 22:20:56.775135 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.775098    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-106555-849b8bdbd5-4trlx"]
Oct 31 22:20:56.776894 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:56.776865    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-106555-849b8bdbd5-4trlx"]
Oct 31 22:20:57.267389 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:57.267293    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-106555-849b8bdbd5-4trlx" podUID="342810a1-1edf-4dde-ac1d-b8844e81d5ac" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.133.0.53:50052\" within 1s: context deadline exceeded"
Oct 31 22:20:57.267389 ip-10-0-136-157 kubenswrapper[4460]: W1031 22:20:57.267354    4460 logging.go:55] [core] [Channel #17078 SubChannel #17080]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.53:50052", ServerName: "10.133.0.53:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.53:50052: operation was canceled"
Oct 31 22:20:57.267389 ip-10-0-136-157 kubenswrapper[4460]: W1031 22:20:57.267355    4460 logging.go:55] [core] [Channel #17077 SubChannel #17079]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.52:50052", ServerName: "10.133.0.52:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.52:50052: operation was canceled"
Oct 31 22:20:57.267389 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:57.267293    4460 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-106555-659c88c8f5-7hdjm" podUID="9fb4edcd-09a0-4f56-9391-574b046c514e" containerName="payload-pre-processing" probeResult="failure" output="timeout: failed to connect service \"10.133.0.52:50052\" within 1s: context deadline exceeded"
Oct 31 22:20:58.223487 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:58.223454    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="342810a1-1edf-4dde-ac1d-b8844e81d5ac" path="/var/lib/kubelet/pods/342810a1-1edf-4dde-ac1d-b8844e81d5ac/volumes"
Oct 31 22:20:58.223859 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:58.223836    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9fb4edcd-09a0-4f56-9391-574b046c514e" path="/var/lib/kubelet/pods/9fb4edcd-09a0-4f56-9391-574b046c514e/volumes"
Oct 31 22:20:58.772310 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:58.772272    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-8p5kz_7fd24cb1-b574-4cfb-8d3c-ee13bae2e73e/limitador/0.log"
Oct 31 22:20:59.732920 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:20:59.732886    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-6f5dcfbf7c-4gxsx_f2ac0597-8657-4e9f-9585-fdac218122b3/manager/0.log"
Oct 31 22:21:00.953022 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:00.952994    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ssl9_26069598-5dc4-4c96-b55f-dd2f09884419/ovn-acl-logging/0.log"
Oct 31 22:21:00.964272 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:00.964249    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4ssl9_26069598-5dc4-4c96-b55f-dd2f09884419/ovn-acl-logging/0.log"
Oct 31 22:21:01.686070 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.686039    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 22:21:01.706656 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.706574    4460 generic.go:356] "Generic (PLEG): container finished" podID="ea04c855-12eb-4628-a947-3b4e8321bef1" containerID="b816bf29bf7a227b1ab47d8eae86b3be3f257d6564a18b8afc3556945da1757f" exitCode=0
Oct 31 22:21:01.706656 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.706630    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"
Oct 31 22:21:01.706845 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.706660    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z" event={"ID":"ea04c855-12eb-4628-a947-3b4e8321bef1","Type":"ContainerDied","Data":"b816bf29bf7a227b1ab47d8eae86b3be3f257d6564a18b8afc3556945da1757f"}
Oct 31 22:21:01.706845 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.706739    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z" event={"ID":"ea04c855-12eb-4628-a947-3b4e8321bef1","Type":"ContainerDied","Data":"adb80e1f73b32803b706087594a9301b0d5b96769d514e54222d0daf6731bd63"}
Oct 31 22:21:01.706845 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.706761    4460 scope.go:122] "RemoveContainer" containerID="b816bf29bf7a227b1ab47d8eae86b3be3f257d6564a18b8afc3556945da1757f"
Oct 31 22:21:01.715463 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.715448    4460 scope.go:122] "RemoveContainer" containerID="9cd7432e1359bfff5b62fbe8a1c9facffb89c649f1993ea889ced7ada81fd433"
Oct 31 22:21:01.723842 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.723813    4460 scope.go:122] "RemoveContainer" containerID="b816bf29bf7a227b1ab47d8eae86b3be3f257d6564a18b8afc3556945da1757f"
Oct 31 22:21:01.724139 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:21:01.724118    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b816bf29bf7a227b1ab47d8eae86b3be3f257d6564a18b8afc3556945da1757f\": container with ID starting with b816bf29bf7a227b1ab47d8eae86b3be3f257d6564a18b8afc3556945da1757f not found: ID does not exist" containerID="b816bf29bf7a227b1ab47d8eae86b3be3f257d6564a18b8afc3556945da1757f"
Oct 31 22:21:01.724209 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.724148    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b816bf29bf7a227b1ab47d8eae86b3be3f257d6564a18b8afc3556945da1757f"} err="failed to get container status \"b816bf29bf7a227b1ab47d8eae86b3be3f257d6564a18b8afc3556945da1757f\": rpc error: code = NotFound desc = could not find container \"b816bf29bf7a227b1ab47d8eae86b3be3f257d6564a18b8afc3556945da1757f\": container with ID starting with b816bf29bf7a227b1ab47d8eae86b3be3f257d6564a18b8afc3556945da1757f not found: ID does not exist"
Oct 31 22:21:01.724209 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.724164    4460 scope.go:122] "RemoveContainer" containerID="9cd7432e1359bfff5b62fbe8a1c9facffb89c649f1993ea889ced7ada81fd433"
Oct 31 22:21:01.724406 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:21:01.724386    4460 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cd7432e1359bfff5b62fbe8a1c9facffb89c649f1993ea889ced7ada81fd433\": container with ID starting with 9cd7432e1359bfff5b62fbe8a1c9facffb89c649f1993ea889ced7ada81fd433 not found: ID does not exist" containerID="9cd7432e1359bfff5b62fbe8a1c9facffb89c649f1993ea889ced7ada81fd433"
Oct 31 22:21:01.724455 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.724415    4460 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cd7432e1359bfff5b62fbe8a1c9facffb89c649f1993ea889ced7ada81fd433"} err="failed to get container status \"9cd7432e1359bfff5b62fbe8a1c9facffb89c649f1993ea889ced7ada81fd433\": rpc error: code = NotFound desc = could not find container \"9cd7432e1359bfff5b62fbe8a1c9facffb89c649f1993ea889ced7ada81fd433\": container with ID starting with 9cd7432e1359bfff5b62fbe8a1c9facffb89c649f1993ea889ced7ada81fd433 not found: ID does not exist"
Oct 31 22:21:01.788159 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.788127    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-credential-socket\") pod \"ea04c855-12eb-4628-a947-3b4e8321bef1\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") "
Oct 31 22:21:01.788328 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.788178    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-istio-envoy\") pod \"ea04c855-12eb-4628-a947-3b4e8321bef1\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") "
Oct 31 22:21:01.788328 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.788235    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-istio-data\") pod \"ea04c855-12eb-4628-a947-3b4e8321bef1\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") "
Oct 31 22:21:01.788328 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.788272    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea04c855-12eb-4628-a947-3b4e8321bef1-istio-token\" (UniqueName: \"kubernetes.io/projected/ea04c855-12eb-4628-a947-3b4e8321bef1-istio-token\") pod \"ea04c855-12eb-4628-a947-3b4e8321bef1\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") "
Oct 31 22:21:01.788328 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.788293    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-workload-certs\") pod \"ea04c855-12eb-4628-a947-3b4e8321bef1\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") "
Oct 31 22:21:01.788328 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.788325    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea04c855-12eb-4628-a947-3b4e8321bef1-kube-api-access-cfjgw\" (UniqueName: \"kubernetes.io/projected/ea04c855-12eb-4628-a947-3b4e8321bef1-kube-api-access-cfjgw\") pod \"ea04c855-12eb-4628-a947-3b4e8321bef1\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") "
Oct 31 22:21:01.788568 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.788365    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-workload-socket\") pod \"ea04c855-12eb-4628-a947-3b4e8321bef1\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") "
Oct 31 22:21:01.788568 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.788402    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ea04c855-12eb-4628-a947-3b4e8321bef1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea04c855-12eb-4628-a947-3b4e8321bef1-istiod-ca-cert\") pod \"ea04c855-12eb-4628-a947-3b4e8321bef1\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") "
Oct 31 22:21:01.788568 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.788443    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-credential-socket" pod "ea04c855-12eb-4628-a947-3b4e8321bef1" (UID: "ea04c855-12eb-4628-a947-3b4e8321bef1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:21:01.788568 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.788472    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ea04c855-12eb-4628-a947-3b4e8321bef1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea04c855-12eb-4628-a947-3b4e8321bef1-istio-podinfo\") pod \"ea04c855-12eb-4628-a947-3b4e8321bef1\" (UID: \"ea04c855-12eb-4628-a947-3b4e8321bef1\") "
Oct 31 22:21:01.788568 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.788514    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-istio-data" pod "ea04c855-12eb-4628-a947-3b4e8321bef1" (UID: "ea04c855-12eb-4628-a947-3b4e8321bef1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:21:01.788853 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.788771    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-workload-certs" pod "ea04c855-12eb-4628-a947-3b4e8321bef1" (UID: "ea04c855-12eb-4628-a947-3b4e8321bef1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:21:01.788913 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.788868    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:21:01.788913 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.788886    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:21:01.788913 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.788902    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:21:01.789101 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.788934    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-workload-socket" pod "ea04c855-12eb-4628-a947-3b4e8321bef1" (UID: "ea04c855-12eb-4628-a947-3b4e8321bef1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:21:01.789180 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.789155    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea04c855-12eb-4628-a947-3b4e8321bef1-istiod-ca-cert" pod "ea04c855-12eb-4628-a947-3b4e8321bef1" (UID: "ea04c855-12eb-4628-a947-3b4e8321bef1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:21:01.790524 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.790493    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ea04c855-12eb-4628-a947-3b4e8321bef1-istio-podinfo" pod "ea04c855-12eb-4628-a947-3b4e8321bef1" (UID: "ea04c855-12eb-4628-a947-3b4e8321bef1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 22:21:01.790658 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.790625    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea04c855-12eb-4628-a947-3b4e8321bef1-istio-token" pod "ea04c855-12eb-4628-a947-3b4e8321bef1" (UID: "ea04c855-12eb-4628-a947-3b4e8321bef1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:21:01.790799 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.790783    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-istio-envoy" pod "ea04c855-12eb-4628-a947-3b4e8321bef1" (UID: "ea04c855-12eb-4628-a947-3b4e8321bef1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:21:01.790837 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.790815    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea04c855-12eb-4628-a947-3b4e8321bef1-kube-api-access-cfjgw" pod "ea04c855-12eb-4628-a947-3b4e8321bef1" (UID: "ea04c855-12eb-4628-a947-3b4e8321bef1"). InnerVolumeSpecName "kube-api-access-cfjgw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:21:01.890375 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.890348    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:21:01.890375 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.890374    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea04c855-12eb-4628-a947-3b4e8321bef1-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:21:01.890540 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.890384    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfjgw\" (UniqueName: \"kubernetes.io/projected/ea04c855-12eb-4628-a947-3b4e8321bef1-kube-api-access-cfjgw\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:21:01.890540 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.890395    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea04c855-12eb-4628-a947-3b4e8321bef1-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:21:01.890540 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.890403    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea04c855-12eb-4628-a947-3b4e8321bef1-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:21:01.890540 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:01.890411    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea04c855-12eb-4628-a947-3b4e8321bef1-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:21:02.055732 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:02.055701    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"]
Oct 31 22:21:02.069253 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:02.069228    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-106555-openshift-default-5bdb5c6bc9-85g6z"]
Oct 31 22:21:02.224569 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:02.224539    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ea04c855-12eb-4628-a947-3b4e8321bef1" path="/var/lib/kubelet/pods/ea04c855-12eb-4628-a947-3b4e8321bef1/volumes"
Oct 31 22:21:02.438391 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:02.438312    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:21:02.548164 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:02.548131    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:21:04.001272 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:04.001238    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:21:05.149232 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:05.149195    4460 scope.go:122] "RemoveContainer" containerID="897b81ab5fda8d89980120ea46da383e207a21c26d76e9796a53e35fba4ea5f6"
Oct 31 22:21:10.090327 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:10.090279    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:21:12.752992 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:12.752954    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-6f5dcfbf7c-4gxsx_f2ac0597-8657-4e9f-9585-fdac218122b3/manager/0.log"
Oct 31 22:21:14.622894 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:14.622859    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"]
Oct 31 22:21:14.623361 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:14.623077    4460 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t" podUID="db9cc38f-cdbc-4ae0-8645-cec895e76033" containerName="istio-proxy" containerID="cri-o://c7a68b6607db9189271bb4488928b5af02890786675d4a05de2b4284ae76d708" gracePeriod=30
Oct 31 22:21:15.504617 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:15.504581    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-65b7c64b77-bhgzk_5d5841bb-6c0c-4771-8742-d7519f0bc582/maas-api/0.log"
Oct 31 22:21:19.793262 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:19.789998    4460 generic.go:356] "Generic (PLEG): container finished" podID="db9cc38f-cdbc-4ae0-8645-cec895e76033" containerID="c7a68b6607db9189271bb4488928b5af02890786675d4a05de2b4284ae76d708" exitCode=0
Oct 31 22:21:19.793262 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:19.790076    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t" event={"ID":"db9cc38f-cdbc-4ae0-8645-cec895e76033","Type":"ContainerDied","Data":"c7a68b6607db9189271bb4488928b5af02890786675d4a05de2b4284ae76d708"}
Oct 31 22:21:19.871466 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:19.871443    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:21:19.962778 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:19.962698    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-istio-data\") pod \"db9cc38f-cdbc-4ae0-8645-cec895e76033\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") "
Oct 31 22:21:19.962778 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:19.962737    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-workload-certs\") pod \"db9cc38f-cdbc-4ae0-8645-cec895e76033\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") "
Oct 31 22:21:19.962995 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:19.962786    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db9cc38f-cdbc-4ae0-8645-cec895e76033-istio-token\" (UniqueName: \"kubernetes.io/projected/db9cc38f-cdbc-4ae0-8645-cec895e76033-istio-token\") pod \"db9cc38f-cdbc-4ae0-8645-cec895e76033\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") "
Oct 31 22:21:19.962995 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:19.962830    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-istio-envoy\") pod \"db9cc38f-cdbc-4ae0-8645-cec895e76033\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") "
Oct 31 22:21:19.963123 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:19.962998    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/db9cc38f-cdbc-4ae0-8645-cec895e76033-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db9cc38f-cdbc-4ae0-8645-cec895e76033-istio-podinfo\") pod \"db9cc38f-cdbc-4ae0-8645-cec895e76033\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") "
Oct 31 22:21:19.963123 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:19.963047    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-istio-data" pod "db9cc38f-cdbc-4ae0-8645-cec895e76033" (UID: "db9cc38f-cdbc-4ae0-8645-cec895e76033"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:21:19.963123 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:19.963102    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-workload-socket\") pod \"db9cc38f-cdbc-4ae0-8645-cec895e76033\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") "
Oct 31 22:21:19.963284 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:19.963097    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-workload-certs" pod "db9cc38f-cdbc-4ae0-8645-cec895e76033" (UID: "db9cc38f-cdbc-4ae0-8645-cec895e76033"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:21:19.963284 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:19.963159    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db9cc38f-cdbc-4ae0-8645-cec895e76033-kube-api-access-4b2zj\" (UniqueName: \"kubernetes.io/projected/db9cc38f-cdbc-4ae0-8645-cec895e76033-kube-api-access-4b2zj\") pod \"db9cc38f-cdbc-4ae0-8645-cec895e76033\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") "
Oct 31 22:21:19.963284 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:19.963194    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-credential-socket\") pod \"db9cc38f-cdbc-4ae0-8645-cec895e76033\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") "
Oct 31 22:21:19.963284 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:19.963237    4460 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/db9cc38f-cdbc-4ae0-8645-cec895e76033-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db9cc38f-cdbc-4ae0-8645-cec895e76033-istiod-ca-cert\") pod \"db9cc38f-cdbc-4ae0-8645-cec895e76033\" (UID: \"db9cc38f-cdbc-4ae0-8645-cec895e76033\") "
Oct 31 22:21:19.963482 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:19.963304    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-workload-socket" pod "db9cc38f-cdbc-4ae0-8645-cec895e76033" (UID: "db9cc38f-cdbc-4ae0-8645-cec895e76033"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:21:19.963482 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:19.963448    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-credential-socket" pod "db9cc38f-cdbc-4ae0-8645-cec895e76033" (UID: "db9cc38f-cdbc-4ae0-8645-cec895e76033"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:21:19.963706 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:19.963648    4460 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:21:19.963706 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:19.963675    4460 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:21:19.963706 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:19.963677    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db9cc38f-cdbc-4ae0-8645-cec895e76033-istiod-ca-cert" pod "db9cc38f-cdbc-4ae0-8645-cec895e76033" (UID: "db9cc38f-cdbc-4ae0-8645-cec895e76033"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 22:21:19.963706 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:19.963689    4460 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:21:19.964076 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:19.963724    4460 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:21:19.965196 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:19.965173    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-istio-envoy" pod "db9cc38f-cdbc-4ae0-8645-cec895e76033" (UID: "db9cc38f-cdbc-4ae0-8645-cec895e76033"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 22:21:19.965196 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:19.965184    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db9cc38f-cdbc-4ae0-8645-cec895e76033-kube-api-access-4b2zj" pod "db9cc38f-cdbc-4ae0-8645-cec895e76033" (UID: "db9cc38f-cdbc-4ae0-8645-cec895e76033"). InnerVolumeSpecName "kube-api-access-4b2zj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:21:19.965480 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:19.965462    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db9cc38f-cdbc-4ae0-8645-cec895e76033-istio-token" pod "db9cc38f-cdbc-4ae0-8645-cec895e76033" (UID: "db9cc38f-cdbc-4ae0-8645-cec895e76033"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 22:21:19.965541 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:19.965478    4460 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/db9cc38f-cdbc-4ae0-8645-cec895e76033-istio-podinfo" pod "db9cc38f-cdbc-4ae0-8645-cec895e76033" (UID: "db9cc38f-cdbc-4ae0-8645-cec895e76033"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 22:21:20.064789 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:20.064754    4460 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4b2zj\" (UniqueName: \"kubernetes.io/projected/db9cc38f-cdbc-4ae0-8645-cec895e76033-kube-api-access-4b2zj\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:21:20.064789 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:20.064784    4460 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db9cc38f-cdbc-4ae0-8645-cec895e76033-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:21:20.064789 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:20.064793    4460 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db9cc38f-cdbc-4ae0-8645-cec895e76033-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:21:20.065038 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:20.064801    4460 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db9cc38f-cdbc-4ae0-8645-cec895e76033-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:21:20.065038 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:20.064810    4460 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db9cc38f-cdbc-4ae0-8645-cec895e76033-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\""
Oct 31 22:21:20.651649 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:20.651613    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-8p5kz_7fd24cb1-b574-4cfb-8d3c-ee13bae2e73e/limitador/0.log"
Oct 31 22:21:20.795731 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:20.795699    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t" event={"ID":"db9cc38f-cdbc-4ae0-8645-cec895e76033","Type":"ContainerDied","Data":"5df93f653ecd7224c08f7511147ddeb22875093e7fdd5764f5b3e064a89c6e3f"}
Oct 31 22:21:20.795731 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:20.795718    4460 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"
Oct 31 22:21:20.796282 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:20.795758    4460 scope.go:122] "RemoveContainer" containerID="c7a68b6607db9189271bb4488928b5af02890786675d4a05de2b4284ae76d708"
Oct 31 22:21:20.865439 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:20.865401    4460 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"]
Oct 31 22:21:20.879196 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:20.879160    4460 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-b659t"]
Oct 31 22:21:21.450827 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:21:21.450786    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:21:21.834463 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:21.834428    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-7dd27_1c463ce3-50ac-473d-bb96-ebc5c3c90e6d/discovery/0.log"
Oct 31 22:21:22.223953 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:22.223866    4460 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="db9cc38f-cdbc-4ae0-8645-cec895e76033" path="/var/lib/kubelet/pods/db9cc38f-cdbc-4ae0-8645-cec895e76033/volumes"
Oct 31 22:21:22.818156 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:22.818126    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58847f9fc5-hcd5s_c0272849-0715-4e09-8d0b-827116fb35d5/istio-proxy/0.log"
Oct 31 22:21:23.390251 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:23.390217    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-hxh82_bb3b6872-a3ff-438d-b1ba-45fca4844b6e/payload-processing/0.log"
Oct 31 22:21:24.503137 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:24.503109    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-bfqc5_678a541a-87a5-4280-b4d4-24ba4f13df8e/main/0.log"
Oct 31 22:21:24.695825 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:24.695793    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-bnv8m_df78aa1c-2468-44bb-a844-6d02c9b319f3/main/0.log"
Oct 31 22:21:24.893306 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:24.893279    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-xcmm7_2c6009fc-0e3d-4178-84f3-eafc8245b15e/main/0.log"
Oct 31 22:21:25.636613 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:25.636578    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-zcv9p_9cd67bf5-8833-4a82-bc33-bed7bdfd4d36/main/0.log"
Oct 31 22:21:35.744544 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:35.744512    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:21:35.855462 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:35.855423    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:21:40.624680 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:40.624645    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9zw4h_22daf2b3-91ca-462b-8714-0e0b0ebbcb88/global-pull-secret-syncer/0.log"
Oct 31 22:21:41.438800 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:41.438768    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log"
Oct 31 22:21:42.131028 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:42.130989    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-157.ec2.internal_6034bd5835de1c87762018f93d657206/haproxy/0.log"
Oct 31 22:21:42.462149 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:42.462064    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:21:43.620975 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:43.620925    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-6f5dcfbf7c-4gxsx_f2ac0597-8657-4e9f-9585-fdac218122b3/manager/0.log"
Oct 31 22:21:48.471876 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:48.471842    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-6f5dcfbf7c-4gxsx_f2ac0597-8657-4e9f-9585-fdac218122b3/manager/0.log"
Oct 31 22:21:48.928009 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:48.927973    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-8p5kz_7fd24cb1-b574-4cfb-8d3c-ee13bae2e73e/limitador/0.log"
Oct 31 22:21:50.777734 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.777696    4460 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bgz52/perf-node-gather-daemonset-r6bdm"]
Oct 31 22:21:50.778195 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778175    4460 cpu_manager.go:414] "containerMap: removing container" podUID="9771611f-879a-44f6-9c1a-f8647dc2d841" containerName="main"
Oct 31 22:21:50.778246 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778196    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9771611f-879a-44f6-9c1a-f8647dc2d841" containerName="main"
Oct 31 22:21:50.778246 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778211    4460 cpu_manager.go:414] "containerMap: removing container" podUID="a22daeb7-8d30-4dcc-98be-e8f404ab1c42" containerName="payload-pre-processing"
Oct 31 22:21:50.778246 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778217    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a22daeb7-8d30-4dcc-98be-e8f404ab1c42" containerName="payload-pre-processing"
Oct 31 22:21:50.778246 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778227    4460 cpu_manager.go:414] "containerMap: removing container" podUID="42f6904e-6141-41a7-a7c5-ee9c766193d2" containerName="cleanup"
Oct 31 22:21:50.778246 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778232    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="42f6904e-6141-41a7-a7c5-ee9c766193d2" containerName="cleanup"
Oct 31 22:21:50.778246 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778242    4460 cpu_manager.go:414] "containerMap: removing container" podUID="b29dc108-b17e-44be-9f2b-1d133f0acfe7" containerName="cleanup"
Oct 31 22:21:50.778246 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778246    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b29dc108-b17e-44be-9f2b-1d133f0acfe7" containerName="cleanup"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778254    4460 cpu_manager.go:414] "containerMap: removing container" podUID="4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" containerName="main"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778258    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" containerName="main"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778265    4460 cpu_manager.go:414] "containerMap: removing container" podUID="dfc291e0-afb6-4fbd-8314-2e2a36096128" containerName="cleanup"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778270    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dfc291e0-afb6-4fbd-8314-2e2a36096128" containerName="cleanup"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778278    4460 cpu_manager.go:414] "containerMap: removing container" podUID="869ed893-352c-4f8f-a174-b42afdeb4de7" containerName="main"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778282    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="869ed893-352c-4f8f-a174-b42afdeb4de7" containerName="main"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778288    4460 cpu_manager.go:414] "containerMap: removing container" podUID="dc0b50fb-08dd-4729-bf9f-01e891bf8ca6" containerName="cleanup"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778292    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc0b50fb-08dd-4729-bf9f-01e891bf8ca6" containerName="cleanup"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778297    4460 cpu_manager.go:414] "containerMap: removing container" podUID="c6686f63-0d95-4d42-8fcb-861e719277ce" containerName="cleanup"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778301    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c6686f63-0d95-4d42-8fcb-861e719277ce" containerName="cleanup"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778308    4460 cpu_manager.go:414] "containerMap: removing container" podUID="342810a1-1edf-4dde-ac1d-b8844e81d5ac" containerName="payload-processing"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778312    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="342810a1-1edf-4dde-ac1d-b8844e81d5ac" containerName="payload-processing"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778318    4460 cpu_manager.go:414] "containerMap: removing container" podUID="8b369e84-9944-4d3b-9bc1-9aa885694d11" containerName="maas-api"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778322    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b369e84-9944-4d3b-9bc1-9aa885694d11" containerName="maas-api"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778330    4460 cpu_manager.go:414] "containerMap: removing container" podUID="db9cc38f-cdbc-4ae0-8645-cec895e76033" containerName="istio-proxy"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778334    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="db9cc38f-cdbc-4ae0-8645-cec895e76033" containerName="istio-proxy"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778340    4460 cpu_manager.go:414] "containerMap: removing container" podUID="dfc291e0-afb6-4fbd-8314-2e2a36096128" containerName="cleanup"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778344    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dfc291e0-afb6-4fbd-8314-2e2a36096128" containerName="cleanup"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778351    4460 cpu_manager.go:414] "containerMap: removing container" podUID="ec53414f-1bc8-4065-819c-d85620195d70" containerName="cleanup"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778356    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ec53414f-1bc8-4065-819c-d85620195d70" containerName="cleanup"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778361    4460 cpu_manager.go:414] "containerMap: removing container" podUID="581aed43-b880-434d-a7ca-3d0c78e54c38" containerName="istio-proxy"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778365    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="581aed43-b880-434d-a7ca-3d0c78e54c38" containerName="istio-proxy"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778370    4460 cpu_manager.go:414] "containerMap: removing container" podUID="e2e149ab-f40b-4d96-91df-b230dfae9e90" containerName="payload-processing"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778376    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e2e149ab-f40b-4d96-91df-b230dfae9e90" containerName="payload-processing"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778387    4460 cpu_manager.go:414] "containerMap: removing container" podUID="2f79cdb1-ad35-4719-8f2b-591fa5965bc5" containerName="istio-proxy"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778395    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2f79cdb1-ad35-4719-8f2b-591fa5965bc5" containerName="istio-proxy"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778405    4460 cpu_manager.go:414] "containerMap: removing container" podUID="2f79cdb1-ad35-4719-8f2b-591fa5965bc5" containerName="istio-proxy"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778412    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2f79cdb1-ad35-4719-8f2b-591fa5965bc5" containerName="istio-proxy"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778427    4460 cpu_manager.go:414] "containerMap: removing container" podUID="5ce9718e-7d7b-4fc4-b506-b25cd880efc3" containerName="payload-processing"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778434    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5ce9718e-7d7b-4fc4-b506-b25cd880efc3" containerName="payload-processing"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778440    4460 cpu_manager.go:414] "containerMap: removing container" podUID="20a6bd1d-39a8-4b3e-898b-bfdd13a155b3" containerName="istio-proxy"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778445    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="20a6bd1d-39a8-4b3e-898b-bfdd13a155b3" containerName="istio-proxy"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778450    4460 cpu_manager.go:414] "containerMap: removing container" podUID="4648d201-9e87-495c-b4aa-cb961fae6f26" containerName="payload-pre-processing"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778456    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4648d201-9e87-495c-b4aa-cb961fae6f26" containerName="payload-pre-processing"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778463    4460 cpu_manager.go:414] "containerMap: removing container" podUID="2412d51b-ffbb-496d-a28e-1a1b3868e98a" containerName="istio-proxy"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778468    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2412d51b-ffbb-496d-a28e-1a1b3868e98a" containerName="istio-proxy"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778477    4460 cpu_manager.go:414] "containerMap: removing container" podUID="48f52b01-f63f-47c2-b0a5-d07370934c67" containerName="payload-processing"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778481    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="48f52b01-f63f-47c2-b0a5-d07370934c67" containerName="payload-processing"
Oct 31 22:21:50.778471 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778487    4460 cpu_manager.go:414] "containerMap: removing container" podUID="876bda78-8ad9-445d-baa7-37c9ab1b4237" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778492    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="876bda78-8ad9-445d-baa7-37c9ab1b4237" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778501    4460 cpu_manager.go:414] "containerMap: removing container" podUID="ea04c855-12eb-4628-a947-3b4e8321bef1" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778505    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ea04c855-12eb-4628-a947-3b4e8321bef1" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778510    4460 cpu_manager.go:414] "containerMap: removing container" podUID="b29dc108-b17e-44be-9f2b-1d133f0acfe7" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778523    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b29dc108-b17e-44be-9f2b-1d133f0acfe7" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778530    4460 cpu_manager.go:414] "containerMap: removing container" podUID="406f2702-7e47-4467-9f9c-a41f8f9ea53d" containerName="payload-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778535    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="406f2702-7e47-4467-9f9c-a41f8f9ea53d" containerName="payload-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778542    4460 cpu_manager.go:414] "containerMap: removing container" podUID="2b6a669d-e3cb-405b-a1c4-6fdfceb78990" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778546    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b6a669d-e3cb-405b-a1c4-6fdfceb78990" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778552    4460 cpu_manager.go:414] "containerMap: removing container" podUID="49303f39-9a79-4b1e-b8dc-12edad88fc96" containerName="payload-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778556    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="49303f39-9a79-4b1e-b8dc-12edad88fc96" containerName="payload-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778564    4460 cpu_manager.go:414] "containerMap: removing container" podUID="b29dc108-b17e-44be-9f2b-1d133f0acfe7" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778569    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b29dc108-b17e-44be-9f2b-1d133f0acfe7" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778575    4460 cpu_manager.go:414] "containerMap: removing container" podUID="9bbd45fa-4d2b-422b-8471-8da4428c1a6f" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778579    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9bbd45fa-4d2b-422b-8471-8da4428c1a6f" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778585    4460 cpu_manager.go:414] "containerMap: removing container" podUID="d141422b-120e-4dc4-bfe0-95ab1391d419" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778589    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d141422b-120e-4dc4-bfe0-95ab1391d419" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778595    4460 cpu_manager.go:414] "containerMap: removing container" podUID="2ce9ce6d-57d6-4085-94e5-2f7f600b81b3" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778599    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2ce9ce6d-57d6-4085-94e5-2f7f600b81b3" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778606    4460 cpu_manager.go:414] "containerMap: removing container" podUID="aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac" containerName="maas-api"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778610    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac" containerName="maas-api"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778618    4460 cpu_manager.go:414] "containerMap: removing container" podUID="982141b1-3b70-4767-9442-264f8920c0b5" containerName="payload-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778626    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="982141b1-3b70-4767-9442-264f8920c0b5" containerName="payload-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778636    4460 cpu_manager.go:414] "containerMap: removing container" podUID="cde3b26f-a0a0-4c0f-835c-cdf613428100" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778642    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cde3b26f-a0a0-4c0f-835c-cdf613428100" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778651    4460 cpu_manager.go:414] "containerMap: removing container" podUID="8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778659    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778670    4460 cpu_manager.go:414] "containerMap: removing container" podUID="ae15a792-70d4-40c4-8222-5a81b6e885be" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778675    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ae15a792-70d4-40c4-8222-5a81b6e885be" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778683    4460 cpu_manager.go:414] "containerMap: removing container" podUID="62d79c06-55b1-4b09-aa24-dd5519154921" containerName="payload-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778687    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="62d79c06-55b1-4b09-aa24-dd5519154921" containerName="payload-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778694    4460 cpu_manager.go:414] "containerMap: removing container" podUID="de079877-5ceb-4370-b8f6-9f6644de83b3" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778698    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="de079877-5ceb-4370-b8f6-9f6644de83b3" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778706    4460 cpu_manager.go:414] "containerMap: removing container" podUID="390c66e4-91b0-4110-92c3-fd754193823d" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778711    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="390c66e4-91b0-4110-92c3-fd754193823d" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778717    4460 cpu_manager.go:414] "containerMap: removing container" podUID="dc0b50fb-08dd-4729-bf9f-01e891bf8ca6" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778721    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc0b50fb-08dd-4729-bf9f-01e891bf8ca6" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778730    4460 cpu_manager.go:414] "containerMap: removing container" podUID="9bbd45fa-4d2b-422b-8471-8da4428c1a6f" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778735    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9bbd45fa-4d2b-422b-8471-8da4428c1a6f" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778741    4460 cpu_manager.go:414] "containerMap: removing container" podUID="aa2bcc59-339d-499c-84f0-b2a6eb365ec0" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778746    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aa2bcc59-339d-499c-84f0-b2a6eb365ec0" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778753    4460 cpu_manager.go:414] "containerMap: removing container" podUID="ec53414f-1bc8-4065-819c-d85620195d70" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778758    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ec53414f-1bc8-4065-819c-d85620195d70" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778764    4460 cpu_manager.go:414] "containerMap: removing container" podUID="390c66e4-91b0-4110-92c3-fd754193823d" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778768    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="390c66e4-91b0-4110-92c3-fd754193823d" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778773    4460 cpu_manager.go:414] "containerMap: removing container" podUID="dd21505e-c4de-43eb-a5da-5c69c7c90785" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778778    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dd21505e-c4de-43eb-a5da-5c69c7c90785" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778784    4460 cpu_manager.go:414] "containerMap: removing container" podUID="876bda78-8ad9-445d-baa7-37c9ab1b4237" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778788    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="876bda78-8ad9-445d-baa7-37c9ab1b4237" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778795    4460 cpu_manager.go:414] "containerMap: removing container" podUID="327e9388-0750-420d-ac21-2ee5fb4224d4" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778799    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="327e9388-0750-420d-ac21-2ee5fb4224d4" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778804    4460 cpu_manager.go:414] "containerMap: removing container" podUID="dfc291e0-afb6-4fbd-8314-2e2a36096128" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778809    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dfc291e0-afb6-4fbd-8314-2e2a36096128" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778814    4460 cpu_manager.go:414] "containerMap: removing container" podUID="b07cb206-7fc8-42a4-9086-dac5ce1d7019" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778818    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b07cb206-7fc8-42a4-9086-dac5ce1d7019" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778826    4460 cpu_manager.go:414] "containerMap: removing container" podUID="ebe9a6f2-adf2-4256-bbea-963048f4a0ad" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778830    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ebe9a6f2-adf2-4256-bbea-963048f4a0ad" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778838    4460 cpu_manager.go:414] "containerMap: removing container" podUID="d68f00ec-71ab-48be-b3bd-d988eeeb5a88" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778842    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d68f00ec-71ab-48be-b3bd-d988eeeb5a88" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778848    4460 cpu_manager.go:414] "containerMap: removing container" podUID="3f60d565-50f1-4b9d-959a-014be6e49128" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778852    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3f60d565-50f1-4b9d-959a-014be6e49128" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778857    4460 cpu_manager.go:414] "containerMap: removing container" podUID="93a42128-ee74-4dcd-b158-27e33cfd44f9" containerName="main"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778861    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="93a42128-ee74-4dcd-b158-27e33cfd44f9" containerName="main"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778867    4460 cpu_manager.go:414] "containerMap: removing container" podUID="d1078aa7-1947-45ce-b6bb-d6550ad61fcf" containerName="main"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778873    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1078aa7-1947-45ce-b6bb-d6550ad61fcf" containerName="main"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778882    4460 cpu_manager.go:414] "containerMap: removing container" podUID="cde3b26f-a0a0-4c0f-835c-cdf613428100" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778889    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cde3b26f-a0a0-4c0f-835c-cdf613428100" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778896    4460 cpu_manager.go:414] "containerMap: removing container" podUID="9a09d750-0cc0-430b-a2d6-6490cf9f1361" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778901    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9a09d750-0cc0-430b-a2d6-6490cf9f1361" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778906    4460 cpu_manager.go:414] "containerMap: removing container" podUID="b07cb206-7fc8-42a4-9086-dac5ce1d7019" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778910    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b07cb206-7fc8-42a4-9086-dac5ce1d7019" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778916    4460 cpu_manager.go:414] "containerMap: removing container" podUID="e2f44b0f-767e-474e-8f9d-9a84dd2cf281" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778920    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e2f44b0f-767e-474e-8f9d-9a84dd2cf281" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778927    4460 cpu_manager.go:414] "containerMap: removing container" podUID="d68f00ec-71ab-48be-b3bd-d988eeeb5a88" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778932    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d68f00ec-71ab-48be-b3bd-d988eeeb5a88" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778954    4460 cpu_manager.go:414] "containerMap: removing container" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778961    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778968    4460 cpu_manager.go:414] "containerMap: removing container" podUID="076218ca-4ba5-43dc-967e-fb30eda6e029" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778973    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="076218ca-4ba5-43dc-967e-fb30eda6e029" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778979    4460 cpu_manager.go:414] "containerMap: removing container" podUID="93b36b86-cb3d-41a0-b3b8-526a57e417da" containerName="maas-api"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778983    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="93b36b86-cb3d-41a0-b3b8-526a57e417da" containerName="maas-api"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.778990    4460 cpu_manager.go:414] "containerMap: removing container" podUID="8a302e86-54e3-4d9f-81ee-350498a9149e" containerName="payload-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779019    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8a302e86-54e3-4d9f-81ee-350498a9149e" containerName="payload-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779028    4460 cpu_manager.go:414] "containerMap: removing container" podUID="9fb4edcd-09a0-4f56-9391-574b046c514e" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779034    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9fb4edcd-09a0-4f56-9391-574b046c514e" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779040    4460 cpu_manager.go:414] "containerMap: removing container" podUID="e438fbbd-062a-4ba7-9c8e-fe954d0cf55b" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779045    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e438fbbd-062a-4ba7-9c8e-fe954d0cf55b" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779051    4460 cpu_manager.go:414] "containerMap: removing container" podUID="42f6904e-6141-41a7-a7c5-ee9c766193d2" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779056    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="42f6904e-6141-41a7-a7c5-ee9c766193d2" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779063    4460 cpu_manager.go:414] "containerMap: removing container" podUID="e97b2260-7cb0-4f61-a862-25b9313beb18" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779067    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e97b2260-7cb0-4f61-a862-25b9313beb18" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779073    4460 cpu_manager.go:414] "containerMap: removing container" podUID="a8c618bb-9eca-482e-8d17-f0f7a1ffd12c" containerName="maas-api"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779077    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a8c618bb-9eca-482e-8d17-f0f7a1ffd12c" containerName="maas-api"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779084    4460 cpu_manager.go:414] "containerMap: removing container" podUID="8ee0ad1f-c884-4875-8af0-b494cf74c092" containerName="maas-api"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779088    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8ee0ad1f-c884-4875-8af0-b494cf74c092" containerName="maas-api"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779095    4460 cpu_manager.go:414] "containerMap: removing container" podUID="c6686f63-0d95-4d42-8fcb-861e719277ce" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779100    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c6686f63-0d95-4d42-8fcb-861e719277ce" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779106    4460 cpu_manager.go:414] "containerMap: removing container" podUID="9bbd45fa-4d2b-422b-8471-8da4428c1a6f" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779110    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9bbd45fa-4d2b-422b-8471-8da4428c1a6f" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779116    4460 cpu_manager.go:414] "containerMap: removing container" podUID="be5c2058-2c58-4394-a4e9-c02e2aa18073" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779120    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="be5c2058-2c58-4394-a4e9-c02e2aa18073" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779126    4460 cpu_manager.go:414] "containerMap: removing container" podUID="15a6d325-3256-4d12-b698-fffafe8c39f0" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779133    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="15a6d325-3256-4d12-b698-fffafe8c39f0" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779143    4460 cpu_manager.go:414] "containerMap: removing container" podUID="869e1829-ba34-4ed7-bd97-37aa003a0d0f" containerName="payload-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779150    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="869e1829-ba34-4ed7-bd97-37aa003a0d0f" containerName="payload-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779160    4460 cpu_manager.go:414] "containerMap: removing container" podUID="dc0b50fb-08dd-4729-bf9f-01e891bf8ca6" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779167    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc0b50fb-08dd-4729-bf9f-01e891bf8ca6" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779178    4460 cpu_manager.go:414] "containerMap: removing container" podUID="cde3b26f-a0a0-4c0f-835c-cdf613428100" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779186    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cde3b26f-a0a0-4c0f-835c-cdf613428100" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779196    4460 cpu_manager.go:414] "containerMap: removing container" podUID="9a09d750-0cc0-430b-a2d6-6490cf9f1361" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779201    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9a09d750-0cc0-430b-a2d6-6490cf9f1361" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779207    4460 cpu_manager.go:414] "containerMap: removing container" podUID="5953786a-52ca-4d85-a60d-a276d6889552" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779212    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5953786a-52ca-4d85-a60d-a276d6889552" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779217    4460 cpu_manager.go:414] "containerMap: removing container" podUID="e2f44b0f-767e-474e-8f9d-9a84dd2cf281" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779222    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e2f44b0f-767e-474e-8f9d-9a84dd2cf281" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779228    4460 cpu_manager.go:414] "containerMap: removing container" podUID="ae15a792-70d4-40c4-8222-5a81b6e885be" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779232    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ae15a792-70d4-40c4-8222-5a81b6e885be" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779237    4460 cpu_manager.go:414] "containerMap: removing container" podUID="4b94e7af-e6fa-434b-b1ac-3069e1d6953e" containerName="revoke-keys"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779241    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4b94e7af-e6fa-434b-b1ac-3069e1d6953e" containerName="revoke-keys"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779248    4460 cpu_manager.go:414] "containerMap: removing container" podUID="afe24009-66f9-4972-9e31-709ec4fa2acb" containerName="payload-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779253    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="afe24009-66f9-4972-9e31-709ec4fa2acb" containerName="payload-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779258    4460 cpu_manager.go:414] "containerMap: removing container" podUID="be5c2058-2c58-4394-a4e9-c02e2aa18073" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779262    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="be5c2058-2c58-4394-a4e9-c02e2aa18073" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779268    4460 cpu_manager.go:414] "containerMap: removing container" podUID="47f0f7a9-1ed4-4046-898f-29220ce6f46b" containerName="payload-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779273    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="47f0f7a9-1ed4-4046-898f-29220ce6f46b" containerName="payload-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779279    4460 cpu_manager.go:414] "containerMap: removing container" podUID="b7d48a8c-34fc-4974-8cc6-1346a27fcc65" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779284    4460 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b7d48a8c-34fc-4974-8cc6-1346a27fcc65" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779366    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="390c66e4-91b0-4110-92c3-fd754193823d" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779375    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="dd21505e-c4de-43eb-a5da-5c69c7c90785" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779380    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="876bda78-8ad9-445d-baa7-37c9ab1b4237" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779386    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="327e9388-0750-420d-ac21-2ee5fb4224d4" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779392    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="dfc291e0-afb6-4fbd-8314-2e2a36096128" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779398    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="b07cb206-7fc8-42a4-9086-dac5ce1d7019" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779407    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="ebe9a6f2-adf2-4256-bbea-963048f4a0ad" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779416    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="d68f00ec-71ab-48be-b3bd-d988eeeb5a88" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779427    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="3f60d565-50f1-4b9d-959a-014be6e49128" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779438    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="93a42128-ee74-4dcd-b158-27e33cfd44f9" containerName="main"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779447    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1078aa7-1947-45ce-b6bb-d6550ad61fcf" containerName="main"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779456    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="cde3b26f-a0a0-4c0f-835c-cdf613428100" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779463    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="c6686f63-0d95-4d42-8fcb-861e719277ce" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779468    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="9a09d750-0cc0-430b-a2d6-6490cf9f1361" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779474    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="b07cb206-7fc8-42a4-9086-dac5ce1d7019" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779478    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="e2f44b0f-767e-474e-8f9d-9a84dd2cf281" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779484    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="42f6904e-6141-41a7-a7c5-ee9c766193d2" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779489    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="d68f00ec-71ab-48be-b3bd-d988eeeb5a88" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779495    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="b9cd1fa6-02c8-42af-b733-8f9c326a5b73" containerName="main"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779501    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="076218ca-4ba5-43dc-967e-fb30eda6e029" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779508    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="93b36b86-cb3d-41a0-b3b8-526a57e417da" containerName="maas-api"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779512    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="8a302e86-54e3-4d9f-81ee-350498a9149e" containerName="payload-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779519    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="9fb4edcd-09a0-4f56-9391-574b046c514e" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779524    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="e438fbbd-062a-4ba7-9c8e-fe954d0cf55b" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779530    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="42f6904e-6141-41a7-a7c5-ee9c766193d2" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779535    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="e97b2260-7cb0-4f61-a862-25b9313beb18" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779541    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="a8c618bb-9eca-482e-8d17-f0f7a1ffd12c" containerName="maas-api"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779546    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="20a6bd1d-39a8-4b3e-898b-bfdd13a155b3" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779552    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="8ee0ad1f-c884-4875-8af0-b494cf74c092" containerName="maas-api"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779562    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="ec53414f-1bc8-4065-819c-d85620195d70" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779567    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="d68f00ec-71ab-48be-b3bd-d988eeeb5a88" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779572    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="c6686f63-0d95-4d42-8fcb-861e719277ce" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779577    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="9bbd45fa-4d2b-422b-8471-8da4428c1a6f" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779581    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="ea04c855-12eb-4628-a947-3b4e8321bef1" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779586    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="be5c2058-2c58-4394-a4e9-c02e2aa18073" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779591    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="15a6d325-3256-4d12-b698-fffafe8c39f0" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779595    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="869e1829-ba34-4ed7-bd97-37aa003a0d0f" containerName="payload-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779600    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc0b50fb-08dd-4729-bf9f-01e891bf8ca6" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779606    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="cde3b26f-a0a0-4c0f-835c-cdf613428100" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779611    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="e2f44b0f-767e-474e-8f9d-9a84dd2cf281" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779617    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="9a09d750-0cc0-430b-a2d6-6490cf9f1361" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779623    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="5953786a-52ca-4d85-a60d-a276d6889552" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779628    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="e2f44b0f-767e-474e-8f9d-9a84dd2cf281" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779632    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="ae15a792-70d4-40c4-8222-5a81b6e885be" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779637    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="4b94e7af-e6fa-434b-b1ac-3069e1d6953e" containerName="revoke-keys"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779642    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="afe24009-66f9-4972-9e31-709ec4fa2acb" containerName="payload-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779652    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="be5c2058-2c58-4394-a4e9-c02e2aa18073" containerName="istio-proxy"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779660    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="47f0f7a9-1ed4-4046-898f-29220ce6f46b" containerName="payload-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779667    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="b7d48a8c-34fc-4974-8cc6-1346a27fcc65" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779679    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="9771611f-879a-44f6-9c1a-f8647dc2d841" containerName="main"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779687    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="a22daeb7-8d30-4dcc-98be-e8f404ab1c42" containerName="payload-pre-processing"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779696    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="b29dc108-b17e-44be-9f2b-1d133f0acfe7" containerName="cleanup"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779703    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="4cf6bb6a-3a46-41f0-9e4f-f0fabbbc82cf" containerName="main"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779708    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="869ed893-352c-4f8f-a174-b42afdeb4de7" containerName="main"
Oct 31 22:21:50.779767 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779713    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc0b50fb-08dd-4729-bf9f-01e891bf8ca6" containerName="cleanup"
Oct 31 22:21:50.786192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779719    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="342810a1-1edf-4dde-ac1d-b8844e81d5ac" containerName="payload-processing"
Oct 31 22:21:50.786192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779724    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b369e84-9944-4d3b-9bc1-9aa885694d11" containerName="maas-api"
Oct 31 22:21:50.786192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779728    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="db9cc38f-cdbc-4ae0-8645-cec895e76033" containerName="istio-proxy"
Oct 31 22:21:50.786192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779733    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="dfc291e0-afb6-4fbd-8314-2e2a36096128" containerName="cleanup"
Oct 31 22:21:50.786192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779738    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="581aed43-b880-434d-a7ca-3d0c78e54c38" containerName="istio-proxy"
Oct 31 22:21:50.786192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779742    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="e2e149ab-f40b-4d96-91df-b230dfae9e90" containerName="payload-processing"
Oct 31 22:21:50.786192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779749    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="2f79cdb1-ad35-4719-8f2b-591fa5965bc5" containerName="istio-proxy"
Oct 31 22:21:50.786192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779754    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="2f79cdb1-ad35-4719-8f2b-591fa5965bc5" containerName="istio-proxy"
Oct 31 22:21:50.786192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779760    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="5ce9718e-7d7b-4fc4-b506-b25cd880efc3" containerName="payload-processing"
Oct 31 22:21:50.786192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779766    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="20a6bd1d-39a8-4b3e-898b-bfdd13a155b3" containerName="istio-proxy"
Oct 31 22:21:50.786192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779771    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="4648d201-9e87-495c-b4aa-cb961fae6f26" containerName="payload-pre-processing"
Oct 31 22:21:50.786192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779776    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="2412d51b-ffbb-496d-a28e-1a1b3868e98a" containerName="istio-proxy"
Oct 31 22:21:50.786192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779783    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="48f52b01-f63f-47c2-b0a5-d07370934c67" containerName="payload-processing"
Oct 31 22:21:50.786192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779789    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="876bda78-8ad9-445d-baa7-37c9ab1b4237" containerName="istio-proxy"
Oct 31 22:21:50.786192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779796    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="ea04c855-12eb-4628-a947-3b4e8321bef1" containerName="istio-proxy"
Oct 31 22:21:50.786192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779801    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="b29dc108-b17e-44be-9f2b-1d133f0acfe7" containerName="cleanup"
Oct 31 22:21:50.786192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779807    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="406f2702-7e47-4467-9f9c-a41f8f9ea53d" containerName="payload-processing"
Oct 31 22:21:50.786192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779813    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b6a669d-e3cb-405b-a1c4-6fdfceb78990" containerName="payload-pre-processing"
Oct 31 22:21:50.786192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779818    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="49303f39-9a79-4b1e-b8dc-12edad88fc96" containerName="payload-processing"
Oct 31 22:21:50.786192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779823    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="b29dc108-b17e-44be-9f2b-1d133f0acfe7" containerName="cleanup"
Oct 31 22:21:50.786192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779829    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="581aed43-b880-434d-a7ca-3d0c78e54c38" containerName="istio-proxy"
Oct 31 22:21:50.786192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779834    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="d141422b-120e-4dc4-bfe0-95ab1391d419" containerName="payload-pre-processing"
Oct 31 22:21:50.786192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779838    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="2ce9ce6d-57d6-4085-94e5-2f7f600b81b3" containerName="istio-proxy"
Oct 31 22:21:50.786192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779844    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="aefc68e6-fb60-4fc3-b791-1c6e0a6bf4ac" containerName="maas-api"
Oct 31 22:21:50.786192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779849    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="982141b1-3b70-4767-9442-264f8920c0b5" containerName="payload-processing"
Oct 31 22:21:50.786192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779854    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b588e9e-1de6-4f7a-b7fd-2f98cde2edf1" containerName="payload-pre-processing"
Oct 31 22:21:50.786192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779860    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="62d79c06-55b1-4b09-aa24-dd5519154921" containerName="payload-processing"
Oct 31 22:21:50.786192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779866    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="de079877-5ceb-4370-b8f6-9f6644de83b3" containerName="payload-pre-processing"
Oct 31 22:21:50.786192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779871    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="9bbd45fa-4d2b-422b-8471-8da4428c1a6f" containerName="cleanup"
Oct 31 22:21:50.786192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779877    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="aa2bcc59-339d-499c-84f0-b2a6eb365ec0" containerName="payload-pre-processing"
Oct 31 22:21:50.786192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.779882    4460 memory_manager.go:365] "RemoveStaleState removing state" podUID="ec53414f-1bc8-4065-819c-d85620195d70" containerName="cleanup"
Oct 31 22:21:50.786192 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.783293    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bgz52/perf-node-gather-daemonset-r6bdm"
Oct 31 22:21:50.804658 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.804625    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b/alertmanager/0.log"
Oct 31 22:21:50.808565 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.808536    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bgz52/perf-node-gather-daemonset-r6bdm"]
Oct 31 22:21:50.842148 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.842104    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/42b6391d-3954-4da5-920a-e902a6334ca7-podres\") pod \"perf-node-gather-daemonset-r6bdm\" (UID: \"42b6391d-3954-4da5-920a-e902a6334ca7\") " pod="openshift-must-gather-bgz52/perf-node-gather-daemonset-r6bdm"
Oct 31 22:21:50.842148 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.842146    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/42b6391d-3954-4da5-920a-e902a6334ca7-proc\") pod \"perf-node-gather-daemonset-r6bdm\" (UID: \"42b6391d-3954-4da5-920a-e902a6334ca7\") " pod="openshift-must-gather-bgz52/perf-node-gather-daemonset-r6bdm"
Oct 31 22:21:50.842374 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.842260    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/42b6391d-3954-4da5-920a-e902a6334ca7-lib-modules\") pod \"perf-node-gather-daemonset-r6bdm\" (UID: \"42b6391d-3954-4da5-920a-e902a6334ca7\") " pod="openshift-must-gather-bgz52/perf-node-gather-daemonset-r6bdm"
Oct 31 22:21:50.842374 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.842309    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/42b6391d-3954-4da5-920a-e902a6334ca7-sys\") pod \"perf-node-gather-daemonset-r6bdm\" (UID: \"42b6391d-3954-4da5-920a-e902a6334ca7\") " pod="openshift-must-gather-bgz52/perf-node-gather-daemonset-r6bdm"
Oct 31 22:21:50.842374 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.842347    4460 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jm49b\" (UniqueName: \"kubernetes.io/projected/42b6391d-3954-4da5-920a-e902a6334ca7-kube-api-access-jm49b\") pod \"perf-node-gather-daemonset-r6bdm\" (UID: \"42b6391d-3954-4da5-920a-e902a6334ca7\") " pod="openshift-must-gather-bgz52/perf-node-gather-daemonset-r6bdm"
Oct 31 22:21:50.883587 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.883552    4460 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bgz52\"/\"default-dockercfg-8gbfk\""
Oct 31 22:21:50.891765 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.891744    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bgz52\"/\"openshift-service-ca.crt\""
Oct 31 22:21:50.891917 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.891874    4460 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bgz52\"/\"kube-root-ca.crt\""
Oct 31 22:21:50.944032 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.943997    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/42b6391d-3954-4da5-920a-e902a6334ca7-sys\") pod \"perf-node-gather-daemonset-r6bdm\" (UID: \"42b6391d-3954-4da5-920a-e902a6334ca7\") " pod="openshift-must-gather-bgz52/perf-node-gather-daemonset-r6bdm"
Oct 31 22:21:50.944219 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.944054    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jm49b\" (UniqueName: \"kubernetes.io/projected/42b6391d-3954-4da5-920a-e902a6334ca7-kube-api-access-jm49b\") pod \"perf-node-gather-daemonset-r6bdm\" (UID: \"42b6391d-3954-4da5-920a-e902a6334ca7\") " pod="openshift-must-gather-bgz52/perf-node-gather-daemonset-r6bdm"
Oct 31 22:21:50.944219 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.944137    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/42b6391d-3954-4da5-920a-e902a6334ca7-podres\") pod \"perf-node-gather-daemonset-r6bdm\" (UID: \"42b6391d-3954-4da5-920a-e902a6334ca7\") " pod="openshift-must-gather-bgz52/perf-node-gather-daemonset-r6bdm"
Oct 31 22:21:50.944219 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.944134    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/42b6391d-3954-4da5-920a-e902a6334ca7-sys\") pod \"perf-node-gather-daemonset-r6bdm\" (UID: \"42b6391d-3954-4da5-920a-e902a6334ca7\") " pod="openshift-must-gather-bgz52/perf-node-gather-daemonset-r6bdm"
Oct 31 22:21:50.944219 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.944166    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/42b6391d-3954-4da5-920a-e902a6334ca7-proc\") pod \"perf-node-gather-daemonset-r6bdm\" (UID: \"42b6391d-3954-4da5-920a-e902a6334ca7\") " pod="openshift-must-gather-bgz52/perf-node-gather-daemonset-r6bdm"
Oct 31 22:21:50.944219 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.944202    4460 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/42b6391d-3954-4da5-920a-e902a6334ca7-lib-modules\") pod \"perf-node-gather-daemonset-r6bdm\" (UID: \"42b6391d-3954-4da5-920a-e902a6334ca7\") " pod="openshift-must-gather-bgz52/perf-node-gather-daemonset-r6bdm"
Oct 31 22:21:50.944219 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.944215    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/42b6391d-3954-4da5-920a-e902a6334ca7-podres\") pod \"perf-node-gather-daemonset-r6bdm\" (UID: \"42b6391d-3954-4da5-920a-e902a6334ca7\") " pod="openshift-must-gather-bgz52/perf-node-gather-daemonset-r6bdm"
Oct 31 22:21:50.944629 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.944361    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/42b6391d-3954-4da5-920a-e902a6334ca7-proc\") pod \"perf-node-gather-daemonset-r6bdm\" (UID: \"42b6391d-3954-4da5-920a-e902a6334ca7\") " pod="openshift-must-gather-bgz52/perf-node-gather-daemonset-r6bdm"
Oct 31 22:21:50.944629 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.944450    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/42b6391d-3954-4da5-920a-e902a6334ca7-lib-modules\") pod \"perf-node-gather-daemonset-r6bdm\" (UID: \"42b6391d-3954-4da5-920a-e902a6334ca7\") " pod="openshift-must-gather-bgz52/perf-node-gather-daemonset-r6bdm"
Oct 31 22:21:50.969127 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.969097    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b/config-reloader/0.log"
Oct 31 22:21:50.969639 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:50.969621    4460 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jm49b\" (UniqueName: \"kubernetes.io/projected/42b6391d-3954-4da5-920a-e902a6334ca7-kube-api-access-jm49b\") pod \"perf-node-gather-daemonset-r6bdm\" (UID: \"42b6391d-3954-4da5-920a-e902a6334ca7\") " pod="openshift-must-gather-bgz52/perf-node-gather-daemonset-r6bdm"
Oct 31 22:21:51.096285 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:51.096205    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b/kube-rbac-proxy-web/0.log"
Oct 31 22:21:51.098150 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:51.098127    4460 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bgz52/perf-node-gather-daemonset-r6bdm"
Oct 31 22:21:51.231344 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:51.231317    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b/kube-rbac-proxy/0.log"
Oct 31 22:21:51.358576 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:51.358489    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b/kube-rbac-proxy-metric/0.log"
Oct 31 22:21:51.398867 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:51.398755    4460 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bgz52/perf-node-gather-daemonset-r6bdm"]
Oct 31 22:21:51.401658 ip-10-0-136-157 kubenswrapper[4460]: W1031 22:21:51.401631    4460 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod42b6391d_3954_4da5_920a_e902a6334ca7.slice/crio-189a63544422436f02f4334915d69b312fca19618b9f2e3677bca763b9c724e1 WatchSource:0}: Error finding container 189a63544422436f02f4334915d69b312fca19618b9f2e3677bca763b9c724e1: Status 404 returned error can't find the container with id 189a63544422436f02f4334915d69b312fca19618b9f2e3677bca763b9c724e1
Oct 31 22:21:51.454552 ip-10-0-136-157 kubenswrapper[4460]: E1031 22:21:51.454522    4460 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-47zb8" containerName="extractor"
Oct 31 22:21:51.485266 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:51.485244    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b/prom-label-proxy/0.log"
Oct 31 22:21:51.618000 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:51.617887    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2fb86d0e-5f7e-48f4-b42c-d5a2096f5a2b/init-config-reloader/0.log"
Oct 31 22:21:51.800279 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:51.800241    4460 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4qsqm_3d9a2e4e-3a81-4ae6-80ed-68542e996259/konnectivity-agent/0.log" limit=1024
Oct 31 22:21:51.894446 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:51.894363    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-r7cc2_0e31eea5-fbe9-47c4-98fa-f2e9af6239e4/kube-state-metrics/0.log"
Oct 31 22:21:51.939318 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:51.939279    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bgz52/perf-node-gather-daemonset-r6bdm" event={"ID":"42b6391d-3954-4da5-920a-e902a6334ca7","Type":"ContainerStarted","Data":"633a3fd3b1b4a00c3163bed68eadbe0c943af48d02857f1da6da92884aaf6ec6"}
Oct 31 22:21:51.939484 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:51.939339    4460 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bgz52/perf-node-gather-daemonset-r6bdm" event={"ID":"42b6391d-3954-4da5-920a-e902a6334ca7","Type":"ContainerStarted","Data":"189a63544422436f02f4334915d69b312fca19618b9f2e3677bca763b9c724e1"}
Oct 31 22:21:51.939484 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:51.939373    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bgz52/perf-node-gather-daemonset-r6bdm"
Oct 31 22:21:51.984014 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:51.983966    4460 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-bgz52/perf-node-gather-daemonset-r6bdm" podStartSLOduration=1.983931406 podStartE2EDuration="1.983931406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 22:21:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 22:21:51.983126043 +0000 UTC m=+3951.899435472" watchObservedRunningTime="2026-10-31 22:21:51.983931406 +0000 UTC m=+3951.900240851"
Oct 31 22:21:52.022211 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:52.022184    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-r7cc2_0e31eea5-fbe9-47c4-98fa-f2e9af6239e4/kube-rbac-proxy-main/0.log"
Oct 31 22:21:52.154806 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:52.154720    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-r7cc2_0e31eea5-fbe9-47c4-98fa-f2e9af6239e4/kube-rbac-proxy-self/0.log"
Oct 31 22:21:52.415623 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:52.415548    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-pnmj4_123bbc02-efa9-4e3d-a00a-7e8c5f35afdc/monitoring-plugin/0.log"
Oct 31 22:21:53.230622 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:53.230591    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-6f5dcfbf7c-4gxsx_f2ac0597-8657-4e9f-9585-fdac218122b3/manager/0.log"
Oct 31 22:21:53.281541 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:53.281517    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-szhn4_d7030dce-004e-405d-8789-014516345b75/node-exporter/0.log"
Oct 31 22:21:53.407259 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:53.407227    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-szhn4_d7030dce-004e-405d-8789-014516345b75/kube-rbac-proxy/0.log"
Oct 31 22:21:53.534134 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:53.534112    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-szhn4_d7030dce-004e-405d-8789-014516345b75/init-textfile/0.log"
Oct 31 22:21:53.668532 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:53.668489    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-kl9j9_d98a0acd-99d1-498f-a483-12ea20459971/kube-rbac-proxy-main/0.log"
Oct 31 22:21:53.800144 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:53.800055    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-kl9j9_d98a0acd-99d1-498f-a483-12ea20459971/kube-rbac-proxy-self/0.log"
Oct 31 22:21:53.928414 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:53.928378    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-kl9j9_d98a0acd-99d1-498f-a483-12ea20459971/openshift-state-metrics/0.log"
Oct 31 22:21:54.909853 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:54.909817    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-xnn8z_8d0b7876-45a7-4c96-be68-26417878ea42/prometheus-operator/0.log"
Oct 31 22:21:55.028260 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:55.028225    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-xnn8z_8d0b7876-45a7-4c96-be68-26417878ea42/kube-rbac-proxy/0.log"
Oct 31 22:21:55.281222 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:55.281191    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7c59959cf4-cgt62_e2461106-96c0-4098-a3e3-d1e71f4ff17c/telemeter-client/0.log"
Oct 31 22:21:55.413800 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:55.413763    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7c59959cf4-cgt62_e2461106-96c0-4098-a3e3-d1e71f4ff17c/reload/0.log"
Oct 31 22:21:55.541680 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:55.541576    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7c59959cf4-cgt62_e2461106-96c0-4098-a3e3-d1e71f4ff17c/kube-rbac-proxy/0.log"
Oct 31 22:21:57.954113 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:57.954086    4460 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bgz52/perf-node-gather-daemonset-r6bdm"
Oct 31 22:21:59.942673 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:21:59.942641    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-954f87865-4q2rj_a6becbb4-8451-4de2-b094-7867363d00ce/console/0.log"
Oct 31 22:22:00.101485 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:22:00.101453    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6dd789c8b7-bwnpt_b6346fef-5922-4ddc-a9dd-d802eeaf29ce/download-server/0.log"
Oct 31 22:22:03.052215 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:22:03.052181    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-7dd27_1c463ce3-50ac-473d-bb96-ebc5c3c90e6d/discovery/0.log"
Oct 31 22:22:03.344611 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:22:03.344530    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58847f9fc5-hcd5s_c0272849-0715-4e09-8d0b-827116fb35d5/istio-proxy/0.log"
Oct 31 22:22:03.535955 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:22:03.535920    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-55fbbbf9dc-hxh82_bb3b6872-a3ff-438d-b1ba-45fca4844b6e/payload-processing/0.log"
Oct 31 22:22:03.736828 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:22:03.736755    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fl2tq_a6207663-1cfa-48d9-8382-12973bbf29fd/dns/0.log"
Oct 31 22:22:03.865611 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:22:03.865585    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fl2tq_a6207663-1cfa-48d9-8382-12973bbf29fd/kube-rbac-proxy/0.log"
Oct 31 22:22:04.752132 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:22:04.752095    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wx67m_bb42ddd8-d246-4bf3-ae35-27aa4584b163/dns-node-resolver/0.log"
Oct 31 22:22:06.277725 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:22:06.277687    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xhzwc_23da9927-8255-4380-a3f0-b7ea9298ad40/node-ca/0.log"
Oct 31 22:22:07.455303 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:22:07.455261    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-8p5kz_7fd24cb1-b574-4cfb-8d3c-ee13bae2e73e/limitador/0.log"
Oct 31 22:22:09.430740 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:22:09.430709    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-7dd27_1c463ce3-50ac-473d-bb96-ebc5c3c90e6d/discovery/0.log"
Oct 31 22:22:09.745389 ip-10-0-136-157 kubenswrapper[4460]: I1031 22:22:09.745304    4460 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58847f9fc5-hcd5s_c0272849-0715-4e09-8d0b-827116fb35d5/istio-proxy/0.log"